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FOREWORD 


The trend toward increased automatic control of aircraft through the use 
of sophisticated on-board digital computers is clearly evidenced by the 
nationwide efforts on active control technology and integration of flight con- 
trol and vehicle configuration. Future high perfomance aircraft, both civil- 
ian and military, will be characterized by an increase in the number of 
sensors, aerodynamic control surfaces, and direct thrust devices used. 

Improved performance achieved through a reduction in weight is being sought 
for large transport aircraft by having active attitude and mode controls allow 
reduced margins of open-loop rigid and flexible stability. Great maneuver- 
ability with adequate ride and handling qualities is be;ing sought for STOL and 
VTOL vehicles through various methods of providing high lift, which usually 
increase the number of actuators to be controlled. 

The increase in functions being sought for aircraft increases the number 
of sensors and actuators on board. The multiplicity of hardware alone induces 
a need for automatic processing ^ of data or of control to aid the pilot. 

The multiplicity of choice that naturally arises also leads to increased com- 
putation and automation. Fortunately, modern control systems theory appears 
to be well suited for the design of such complex systems. When the critical- 
ity of survival of some of these functions is considered, however, the picture 
changes: there is a great increase in hardware, some increase in choice, but 

not yet a corresponding growth in constructive and relevant theory. Neverthe- 
less, the new organizational and functional questions are control and systems 
theoretic in nature and should be addressed by the control system community. 

A workshop bringing about 30 government, university, and industry 
researchers to encourage the necessary dialog was organized by MIT’s Elec- 
tronic Systems Laboratory, sponsored by Ames Research Center, NASA, and held 
at MIT August 18-20, 1975. It consisted of formal and informal presentations 
of o surveys and theoretical and practical developments and of discussions. 

This report contains the formal proceedings of that workshop. 

The first group of papers relates to implementing advanced systems. 

After Doolin’s introduction pointing out the opportunities for control-system 
theoretic contributions to future flight control, Taylor reviews NASA's 
active control program. The review points out the performance benefits of 
active control as well as the stringent reliability requirements. Next Meyer 
describes a series of experiments planned at Ames Research Center for V/STOL 
(vertical or short takeoff and landing) aircraft. These experiments are to 
determine the efficacy of a new versatile digital-computer-oriented flight- 
control system for the various active control modes associated with V/STOL 
flight. Another approach to solving the problem of a flexible digital flight- 
control system is given next by Stein (which was delivered by Yore). It, too, 
alms ait., satisfying the requirements of operating throughout a flight envelope 
with a structure that can allow extra provisions for reliable operation. In a 
similar vein, Astroni discusses some of the research carried out at the Lund 
Institute in Sweden on self-tuning regulators and their experimental use in 
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controlling sea ships. Montgomery's paper, the final one of this group, 
addresses the problem of failure detection and the computational experiments 
being conducted at Langley Research Center for. feasible implementation of 
maximum likelihood methods. 

Two informal presentations are not included in these proceedings. 

Berger discussed (see ref. 1) the present approach to reliable operation as 
employed by McDonnell-Douglas. Requirements are satisfied by quadruple 
redundancy of hardware. Particular advantage is taken of built-in test equip- 
ment. In some cases, this "firmware” detects degradation in hardware perfor- 
mance and alerts the flight crew or maintenance accordingly. In a second 
informal discussion, Clark (ref. 2) recounted experiences of using observers 
to detect incipient failures in inertial instruments. 

Saridis' paper on learning systems provides a transition between the 
practically oriented papers above and the theoretical discussions to come. 

This paper, together with that of Tse and an informal discussion by Ho of 
team decision theory, provided an excellent view of the status of theory and 
heuristics in combined estimation and control. Learning is used to develop 
an input-output map when the intervening system is assumed to be Inexplicably 
complex. Dual control (discussed next) deserves consideration when the mis- 
sion is sensitive to precise knowledge of parameters and the parameters in 
turn can best be determined if the system is perturbed. Team problems model 
decentralized control and decentralized information where different control- 
lers share the same goal. The formidable difficulties in dual control and 
team problems are illustrated by the paucity of definite quantitative results 
to date. 

The extensions to gain margin theorems in Athens' paper derive from con- 
siderations of decentralized control and robustness research currently under 
investigation in the Electronic Systems Laboratory. Kleinman's paper reviews 
the successes and promise of modern control systems theory in modeling human 
behavior. The text for the presentation is not available, but the figures 
that are reproduced herein substantially convey the information. The final 
paper along control systems theory lines is Wonham's review of the results in 
linear multivariable theory obtained both by his geometric approach and by 
the transfer matrix approaches of Wolovlch and Rosenbrock. These proceedings 
conclude with two excellent reviews of current research efforts in nonlinear 
filtering and failure detection by Rhodes and Willsky. 

The discussion periods emphasized how very complicated software becomes 
as soon as an operation is automated. Lacking the great tolerance to error 
that a human operator provides means that the conditions of operation must be 
specified in extreme detail. It was recognized that the required automation 
would be unthinkable except for digital computation. 

Although hardware people did not attend the workshop, their claim to 
better reliability than software can give was voiced. Clearly, software- 
generated functional reliability will not replace all hardware replication, 
yet software and computation reliability is being improved. The confidence 
in the reliability of hardware operation is largely due to the increased use 
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of relatively simple software or "firmware” as in builtin test equipment. 
Hence the argument of hardware vs. software is really one of degree. It is 
an argument that cannot be resolved definitively as yet because we do not 
now have appropriate principles by which to organize the logical structure of 
the software. 

As principles of organization for the synthesis of functional reliabil- 
ity are missing, so are criteria and methods for its analysis. Available 
failure modes assessment and event tree methods, of course, provide much 
information. They do not give rise to a dynamics that would enable a con-^ 
venient process for optimization or tradeoff studies. Berger recalled that 
Honeywell had once begun a study that could have led to a dynamic finite- 
state space method of analysis (ref. 3). Apparently the work was discon- 
tinued. 

Participants on the aeronautics side had the benefit of an excellent 
review of the status of most control system theory. Participants on the 
control systems side got an excellent perspective on expectations and open 
problems in aeronautics. People of all backgrounds represented joined in the 
lively discussions. If this dialog concerning principles and methods of 
designing for reliability persists, it will be to the great benefit both of 
control-system theory and of aeronautics, and the workshop will have served 
its purpose. 

This workshop was made possible through NASA grant NSG-2076. We are 
grateful to Mr. R. A. Osborne, Ms. B. Peacock, and Mr. W. H. Lee for their 
tireless efforts in organizing the workshop and for keeping everybody happy. 


Michael Athens Brian F. Doolin 

Director, MIT Electronic Systems Laboratory NASA Ames Research Center 
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INTRODUCTION 


Brian F. Doolin 
Ames Research Center, NASA 


My thanks to all of you for coming to participate in what promises to be 
a stimulating and productive workshop. My particular thanks to our chairman, 
Mike Athans, for his recommendations that spawned this workshop and for his 
labors that brought it about. 

My purpose this morning is simply introductory and motivational. The 
workshop's title, "Systems Reliability Issues for Future Aircraft," covers a 
call for help in the task of using systems theory to improve the reliability 
of aircraft flight. What is sought is direction in developing a unified ana- 
lytic approach to exploit the couplings between the many sources of informa- 
tion and between the many sources of control in airplanes. 

Much of the attention in Control Systems Engineering has focused on 
developing what is by now an extensive and effective theory to analyze and 
synthesize systems that are relatively simple structurally. The success of 
that theory, together with the improvements evident in the technology of 
digital computation, emboldens us to consider directly the challenge thrust 
by aeronautics. The source of the challenge is the increase in the number of 
control functions being demanded of many types of aircraft. 

Control Systems Engineering must find a rational approach to structuring 
these control functions so as to secure both order in their increased com- 
plexity and flexibility in their use. The task is important in that its suc- 
cess will allow other aircraft technologies to achieve their goals of perfor- 
mance and economy by reducing the replication of components that reliability 
considerations will otherwise require. 

For motivation, I will point out opportunities for control that some 
aircraft provide, and lean on a little history to show the interplay between 
human and automatic operation. Many of you are intimately familiar with 
these topics and will discuss some of them to much greater depth. I ask your 
forbearance. 

Figures 1 to 4 are of aircraft or aircraft models that have been flown 
or tested at Ames. They illustrate various opportunities for aerodynamic 
or propulsive control. Figures 1 and 2 show some aerodynamic controls. 

Figure 1 shows our C-141 aircraft with its flaps, at the trailing edge of the 


wings, partially deployed — the flap is split into two sections. The 
ailerons that make the aircraft roll are located farther along the trailing 
edge of the wing. Besides their primary job of rolling the aircraft or 
increasing its lift at the lower flight speeds, these surfaces can be used 
to shape the airloads on the wing. Leading-edge surfaces are also used. For 
various reasons, the vertical control surface of the tail is sometimes split. 

Some aircraft also have small surfaces — canard surfaces — on the for- 
ward part of the fuselage. An example is the wind-tunnel model in figure 2. 
They are used to help control body bending or even to improve aircraft con- 
trol at low speeds. Similar controls have been installed on special aircraft 
to generate controllable side forces. 

Perhaps figure 3 portrays the propulsion man’s dream of what aircraft 
should look like: a lift-fan V/STOL vehicle, models of which have undergone 

wind-tunnel testing. Each of six jet engines drives a turbine that sucks in 
air. The two rear engines have shrouds that deflect the air downward during 
flight at low speeds and hover. They are the only engines operating in 
normal flight. Louvers below the fans modulate the flow for attitude control 
at speeds too low for normal aerodynamic control. You can well imagine that 
piloted digital simulation models have been used to investigate various 
engine failures and the ability of pilots to handle the failures. 

Figure 4 shows the Augmentor Wing Research Aircraft, These flaps are 
not ordinary aerodynamic surfaces, but ducts containing a flow of cool air 
from the engines. This flow also entraps air flowing over the wing. Perhaps 
you can see this nozzle on the engine. There are four of them in all, two 
on each engine. They can be rotated to deflect the direct hot air flow of 
the engines from straight back to straight down and even somewhat forward. 

The combination of direct cold engine air, entrapped flow over the wing, and 
directed thrust gives enough lift for the aircraft to descend steeply but 
gently at 80 or 90 mph. These photographs are shown to illustrate that an 
increasing number of aerodynamic and propulsive controls are evident on 
present and future aircraft. 

If the pilot were responsible for using all these controls, he would be 
faced with enormous tasks of actions and decisions. Such regards have been 
the basic cause for considering automation on aircraft. Figure 5 summarizes 
the basis qualitatively: If things happen too fast for the pilot to handle, 
or if there are just too many things to keep track of and make judgments on, 
then it is necessary to give him relief. On the other hand, if he only has 
minor functions to fiddle with over an extended period of time, he gets bored 
and calls for help. 

Figure 6 illustrates the time-scale spectrum of operations associated 
with flight. The middle column refers to the aircraft, and that on the right 
to the pilot. The durations of flight operations are from the order of hours 
for the cruise portion of flight through mission phases like takeoff and 
approach and landing that take minutes. Duration of flight regimes or mis- 
sion phases overlaps the duration of flight-path dynamics whose time con- 
stants are minutes to seconds. Then attitude dynamics takes over with short 
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periods of 1 to 2 sec. Structural effects overlap here with limit cycle 
flutter periods reaching down to fractions of a second. The spectrum gives 
an understandable ordering of opei;ation: each shorter period motion (except 

for the structural noise) acts as the driving function for the next slower 
motion. 

The human response is /characterized in the next column, which shows that 
the pilot *s performance deteriorates on both sides of periods of the order of 
minutes. What could not be shown on the figure is a plot of human activity 
vs. operational phase how many things are to be done or decisions to be 
made. Those functions depend on failure conditions (whose response times 
fall in the lower part of the time scale) and on the design of the control 
and Information systems, including help given the pilot by automatic systems. 

The pilot wants control where he is most effective so long as the load 
is reasonably organized and not excessive. He not only wants control, but 
the total package is most effective with him operating in his best region, 
as the little^ history of the early days of flight (fig. 7) illustrates 
(ref. 1). 


The first time period in the figure corresponds to the earliest days of 
this century when the vehicle was being developed as a tool for transporta- 
tion. Flights were short and limited to the best weather. As flight times 
increased, navigation followed roads and available maps. Since you cannot 
go very far or often without running into weather giving poor visibility, 
the next pilot aids had to do with his receiving information via radio, 
first voice and then also beacon signals that provided the earliest skyways. 
With improved vehicles and trip times, the pilot no longer had to pay con- 
stant attention to routine and dull tasks since autopilots now could maintain 
a status quo. 

The corresponding history of vehicle stabilization and control is con- 
veyed in the second part of figure 7. A major contribution of the Wright 
brothers was to relax the inherent stability that earlier designers were 
building into their vehicles. When the pilot was allowed to assume some of 
the stabilization duties, path control became reasonable. Stabilization 
required too much agility and attention — it was exhausting. So Sperry 
relieved the pilot of some of this task with his gyrostabilizer in 1913. He 
improved stabilization while retaining the improved configuration technology. 
This mechanism was done away with once the existence of a tradeoff between 
stability and control was understood and airframe designers built compromises 
into the configuration. The compromises were haphazard until the mid-thirties 
when researchers conceived of handling qualities as a design guide and began 
to regularize design conditions. 

Figure 8 brings us up to date. Our present highly developed airway 
system that enabled long-distance flight began with the introduction of 
simple ground beacons. Improvements in onboard instrumentation and computa- 
tion have recently relaxed the need for strict following of the one-dimensional 
airway system. Adding one degree of freedom in flight — two-dimensional, or 
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area, navigation — is becoming commonplace. In fact, at Ames, four-dimensional 
feasibility has been demonstrated. 

For the remaining items listed in the figures, onboard capability and 
requirements really increase, each requirement calling for improvements and 
additions in controls and sensing. All the while, built-in means of supple- 
menting reliability become more important until, with relaxed static stabil- 
ity, we are back to the Wright brothers, and reliability must be superimposed 
and guaranteed. All these requirements and more are being satisfied, as some 
of you will be telling us over the next day or so. 

Now comes the problem of coping with the complexity of control and the 
amount of Intelligence oabout status and environment. The onboard system must 
understand what is occurring, anticipate difficulties, and avoid them. 

The issues of reliability and redundancy are being addressed from the 
systems point of view, as the brief list in figure 9 indicates. Broen 
(ref. 2) has suggested improvements in handling voting smoothly and in not 
destroying form when lines of Information are lost. The advantages of improved 
use of information made available by modern estimation methods were discussed 
by Beard (ref. 3). Montgomery and Price (ref. 4) and Willsky and Jones 
(ref. 5) are making this approach a reality, as well as teaching us how 
complex the task can be. Martin and Harding (ref. 6) investigated the type 
of redundant control occurring in the autopilot design for the augmentor wing 
and found that strategy of control can be partitioned, a result that provides 
some basis for a formal division of control authority and also allows backup 
or redundancy. 

Certainly, the beginnings of the intelligent use of available redundancy 
to reduce the need for replication for a given level of reliability are found 
in these works of Beard, Montgomery, Willsky, Martin, and others. The systems 
theoretic basis they use is available. For this type of approach to encom- 
pass the total flight-control problem and coordinate available resources in 
the face of increasing control and sensing requirements, additions to the 
theory are required. The time is right for these additions now that systems 
theorists are beginning to pay attention to the structure of large-scale 
systems. 

Flight control seems to provide a good model for a large-scale system 
because it is more amenable to analytic description than most other examples 
available. We in the aeronautics business propose that flight control be 
used as a paradigm, of course, since we are interested in developing the 
proper theory. Our interest is evidenced by a workshop held last year at 
Utah State University (ref. 7). The main thrust of that workshop was to seek 
a common description of large-scale systems. 

At the workshop, the best criteria advanced for formally treating prob- 
lems as large were those that have historically driven us to automatic con- 
trol, namely, the first three reasons shown in figure 5. The many decisions 
and operations called for in large systems make it plausible that considera- 
tion be given to hierarchic structures for their design. The theoretical 
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papers of that workshop aimed at building a proper system model by first 
postulating a hierarchic or at least an aggregated structure. Some authors 
inquired into the information and control patterns that conserved that struc- 
ture. Others investigated conditions that preserve some overall stability 
when interconnections in that structure were torn. But no formalized basis 
for aggregating or for generating hierarchic structures appeared. Finding it 
may help uncover a formal approach to a flexible organization we are aiming 
for. I anticipate the deliberations of the next few days, knowing that, 
directly or indirectly, they will evoke the conceptual framework to design a 
reliable control system. 
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APPLICATION OF ACTIVE CONTROLS TO CIVIL TRANSPORT AIRCRAFT 


Lawrence W. Taylor, Jr. 
Langley Research Center, NASA 


INTRODUCTION 


Active controls had its start before the Wright Brothers flew at 
Kitty Hawk in 1903. In 1894, Hiram Maxim (ref. 1) designed an active control 
system that used steam-powered gyros connected to steam-powered actuators to 
stabilize his airplane. The tests he conducted on a constrained rail system 
which enabled the airplane to go forward but not to lift off met with disaster 
as the lift created was great enough to break away from the rail system, 
destroying the aircraft before it could be flown. Most aircraft since have 
relied on inherent stability, with automatic systems being used to improve 
handling qualities. A few aircraft, however, have required automatic control 
systems to enable the pilot to fly the airplanes (i.e., the X-15, B-58, and 
SR71 airplanes). On today’s civil aircraft, certain devices such as yaw 
dampers and speed stabilizers are required for the safe operation of the 
aircraft. On the L-1011, for example, the yaw damper must be operable before 
the airplane can be dispatched for its flight. Over the past several years, 
the Flight Djmamics Laboratory at Wright-Patterson AFB has promoted the 
concept of the control configured vehicle (CCV) whereby the control consider- 
ations are considered to be an integral part of the preliminary design of an 
aircraft (ref. 2). It is in this spirit that active controls are being 
investigated for civil transport aircraft by NASA. NASA has funded a series 
of design studies (refs. 3-5) in which benefits of active controls were 
addressed. There remains, however, the task of studying in depth the impact 
of active controls on civil transport aircraft. It is not the purpose of 
this paper to give a complete dissertation on the application of active con- 
trols but rather to describe some of the very complex problems involved in 
integrating active controls and to give some notions as to an approach. It 
is hoped that some of these notions might prove useful to the aircraft 
designers whose job it is to "put it all together." Figure 1 is a schematic 
of the job of integrating propulsion, aerodynamics, structures, and controls 
in a conceptual design. References 6 through 8 describe many of the design 
considerations involved with active controls not discussed here. 


ACTIVE CONTROLS 


Active controls consists of several concepts (figs. 2 and 3), each of 
which improves some facet of airplane performance. For example, the reduoed 
static stability system uses a servoed horizontal stabilizer to provide 
stability as opposed to a larger fixed horizontal stabilizer. By reducing 
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the size, weight, and drag of the tall, the airplane can be made lighter and 
more efficient. It also enables a smaller wing to be used since the horizontal 
stabilizer can now supplement the wing lift. In a guet toad atteviation aye- 
teniy flaps and the horizontal stabilizer are actuated to reduce the peak 
dynamic loads resulting from gusts. The result Is that a lighter wing struc- 
ture can be used, thereby reducing the total weight of the airplane. In much/ 
the same way, the maneuver toad atteviabion system reduces wing root bending 
by causing a lift distribution more concentrated toward the center of the 
airplane. Again, the reduced structural weight results In Improved airplane 
efficiency. The flutter suppression system stabilizes aeroelastlc modes that 
otherwise might flutter by the judicious actuation of control surfaces at 
strategic locations on the airplane. When applied to a new airplane concep- 
tual design, all of these functions will likely be merged Into a single active 
control system. The Individual concepts are useful In synthesizing the control 
law and In working trade studies during the early design phase. 

Another consideration of active controls Is the degree to which reliance 
Is placed on the proper functioning of the control system. Active controls 
can be used In a noncrltlcal function In which case a failure would not be 
catastrophic or they can be used In a flight critical role so that their 
successful operation Is required to operate the airplane. The reliability, 
complexity, and cost of control systems having a flight critical role may 
detract from benefits In efficiency that would otherwise be realized. One 
can perhaps begin to appreciate the very complex trades yet to be analyzed In 
detail before one can say with assurance what role active controls should play 
In future civil transports. 


NASA ACTIVE CONTROL TECHNOLOGY PROGRAM 


The objective of the NASA Active Control Technology Program (fig. 4) Is 
to Identify, develop, and validate the controls technology required to 
Integrate active controls In future subsonic civil aircraft. The program 
(fig. 5) consists of four parts: (1) baseline data that establish data and 

certain techniques required to apply active controls, (2) validation flight 
and wind-tunnel tests to assess the accuracy of certain theoretical results, 
(3) an active controls analysis package that Incorporates the necessary 
analytical techniques Into a versatile set of computer programs, and (4) Inte- 
grated conceptual design studies Involving active control. The active control 
technology deemed to be ready by a time such as 1985 will then be directed 
toward suitable flight demonstrations In preparation for the application of 
flight critical active controls on future civil transports. 

This paper discusses the approach being taken by NASA as part of the 
Active Control Technology Program to Integrate active controls In the 
conceptual design phase. The approach Is analytical and Is depicted by 

• A mathematically expressed design criterion such as return on total 
Investment 
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• Mathematically expressed relationships and constraints involved in the 
conceptual design process 

• A mimerical technique that optimizes the design variables with respect 
to the criterion 

• Manual interface to assess discrete alternatives involved in the design 


DESIGN CRITERION 


The design criterion should be the same for all aspects of the airplane 
and should reflect the basis on which the aircraft will be judged in terms of 
its performance. It must be expressible in mathematical terms and must be a 
scalar. The design criterion should merge (fig. 6) all of the various con- 
cerns of the design, whether these be performance, cost, revenue, safety, 
noise, pollution, or fuel consumption. Criteria used in the past include 
direct operating cost, sometimes modified to reflect revenue and depreciation, 
and return on an investment. Return on investment is of particular interest 
to the author since ROI is believed to be close to that quantity in which the 
merit of a civil transport airplane is judged by its user, the airline. For 
our purposes, the design problem is considered in terms of determining the 
values of the numerous design variables that minimize the design criterion — 
return on investment. Many equality and inequality constraints must also be 
satisfied. 

A very useful concept concerning the design criterion is what might be 
called the "linearized design criterion" (fig. 7). If the design criterion 
is thought of as a function of the many design variables of the airplane, it 
is possible to differentiate the design criterion (in concept at least) with 
respect to these design variables. If it were possible to do this, one would 
obtain a linear version of the design criterion. A single example is 

AROI = initial cost/ ($10 M) + operating cost/ ($126/hr) + weight/ (5657 lb) 

Each unit that AROI is increased approximately corresponds to an Incremental 
change in ROI of about 1 percent of return, for the current class of wide-body 
jets. This linearized design criterion can then be used independently by 
various design specialists to aid in the assessment of certain alternatives 
during the design process. In this way, numerous design studies can be per- 
formed using the same overall criterion. As a particular conceptual design 
progresses, the coefficients of the linearized design criterion should be 
reassessed. 


ACTIVE CONTROL ANALYSIS PACKAGE 


The collection of computer programs that embodies the active control 
design approach being proposed here is called the Active Control Analysis 
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Package. The major groupings shown in figure 8 are sizing, modeling, control, 
response, and adjust modules. The design philosophy is to represent the 
interests and constraints of all participants by mathematical relationships 
as compared to human participants (ref. 9). Implementation of such a notion 
is still in its infancy and much more work remains to be done before the goals 
of the Active Control Analysis Package can be achieved. A major contribution 
to ACAP is expected to be a portion of the program being developed by Aerospace 
Systems Incorporated (ref. 10). 


Sizing Module 

This part of the program involves aerodynamic, control, and response 
modules. The aircraft sizing module of the analysis package consists of little 
more than the fundamental relationships contained in every conceptual design 
of a subsonic civil transport. The only new thing is that the procedure has 
been computerized and a definitive criterion is maximized automatically. 
Provisions have also been made to include the effects of active control 
functions. A technique for estimating the costs and weights of active controls 
is being developed by Boeing (Contract NASl-14064) . The results of this work 
are incorporated in the sizing module. The sizing module continues to be 
developed, but already it has proved to be useful. The aircraft sizing 
module was the means by which the previously discussed linearized design 
criterion was determined. It has also been used to assess the benefits of 
active controls, the supercritical wing, composite structures, and laminar 
flow control. Figure 9 shows the results of the sizing program for airplane 
configurations that either maximize ROI or seat miles per gallon. The inci- 
dental improvement in seat miles per gallon due to the supercritical wing and 
active controls is about 6 and 12 percent, respectively. The ground rules 
used to compute the comparison are given in table I. 


Modeling Module 

The dynamic modeling portion of the Active Control Analysis Package 
includes making linear constant coefficient state space models that Include 
static and unsteady aerodynamics, structural dynamics, sensor dynamics, and 
control compensation into a single equation of the form x = Ax + Bu. The 
streamlined doublet-lattice technique is presently being used for the unsteady 
aerodynamics with an alternative module using Morino aerodynamics (ref. 11) 
soon to be available. Comparisons of these methods for determining the 
unsteady aerodynamic characterisitcs are about to start. The advantage of 
having the unsteady aerodynamics in the state variable format is that it 
enables a single mathematical model to be used for analysis of both high- 
frequency phenomenon such as flutter and low-frequency "rigid-body” motion. 

A control law can thus be synthesized without having to make a special 
investigation of flutter stability. Figure 10 shows an example of a root 
locus of dynamic pressure, where the critical dynamic pressure for flutter 
was accurately determined by the state variable model. The results that are 
almost Identical to those obtained by frequen:y domain techniques indicate 
the validity of the state variable model at high frequencies. The dynamic 



model was obtained using a technique pioneered by personnel at the Boeing 
Airplane Company, The results shown in figure 10 use programs developed at 
the Langley Research Center by Irving Abel and Ray Hood. The specific calcu- 
lations were performed by Mr. Hood. 


Control Module 

The approach taken to synthesize the various active control laws are 
outlined in figure 11. For maneuver load control, feedforward compensation 
will be used to the extent possible. The purpose is to minimize primarily the 
wing weight by reducing wing bending moment due to the 2-1/2 g maneuver 
required of civil transport aircraft. By employing feedforward, the stability 
of the airplane is not affected and numerous feedback problems are avoided. 

Only in the event of considerable uncertainty of the plant characteristics 
will feedback be used. For gust load alleviation and ride quality control, 
the approach is to measure the gust as accurately as possible and feed this 
forward as well. There will be a frequency range (probably a few Hertz) 
beyond which the accuracy of the measured gusts will be too poor to be of any 
use. At these higher frequencies, it will be necessary to desensitize the 
aircraft response through feedback designed for this purpose. Feedback syn- 
thesis presents a particularly difficult challenge, however. Because of the 
extremely large number of state variables involved (fig. 12), reduced state 
feedback is necessary. A couple of approaches to synthesizing reduced state 
control laws are being pursued. Figure 13 is a diagram in which stochastic 
models of both disturbances and desired responses are used to formulate a 
quadratic minimization problem. An algorithm for solving this problem for 
constrained feedback and feedforward matrices is being developed by Dr. Howard 
Kaufman of Rensselaer Polytechnic Institute (NASA Grant NSG-1188) . An alter- 
nate approach is being developed by Dr. Isaac Horowitz, University of Colorado 
(NASA Grant NSG-1140) . Dr. Horowitz has developed a computer program that 
solves the difficult synthesis problem which minimizes the bandpass for given 
levels of uncertainty in the plant parameters and a particular system perfor- 
mance requirement. It is hoped that this solution for the single input/single 
output problem will serve as a building block for more complex practical 
problems. A quadratic cost function will be used involving the mean-square 
acceleration at various stations of the aircraft. For reduced state feedback, 
a quadratic cost function based on pilot ratings and passenger comfort will be 
employed. Figure 14 shows some preliminary results in expressing pilot rat- 
ings as a quadratic performance index suitable for control synthesis. Similar 
results have been obtained for the pitch degrees of freedom. The work is being 
being done by Dave Middleton of Langley Research Center. Reduced state feed- 
back will be required because of the complexity of the aeroelastic modes. 
Flutter suppression and structural mode control will include both aeroelastic 
modes and unsteady aerodynamics in models used for control synthesis to assure 
stability and long structural life. All systems are considered at various 
levels of complexity and in several flight and loading conditions. The 
linearized design criteria will be used to assess candidate system configura- 
tions considering not only their system performance but their reliability and 
maintainability as well. 
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Response and Adjust Modules 

The response module computes the response of all aspects of the airplane 
being considered-. These responses are compared with constraints not contained 
In the sizing or other modules. Adjustments are then made based on this com- 
parison. This loop Is Iterated until convergence Is obtained, resulting In a 
design that satisfies all constraints and maximizes the design criterion. 


CONCLUDING REMARKS 


The problem of Incorporating active controls In the conceptual design of 
a civil transport Is a very complicated and Involved problem. It Is a problem 
that requires our best understanding and sophisticated computer programs to 
assist us In the very complex trade studies required. The purpose of the 
NASA Active Control Technology Program Is to contribute to the solution of 
this task by developing an Active Control Analysis Package. The package will 
not only add to the analysis tools available but will organize them to greatly 
facilitate conceptual design studies. 
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TABLE I.- PARAMETER VALUES USED IN COMPARISON OF TECHNOLOGY BENEFITS 




Trim drag 

Skin friction 

Tail area 

Maintenance 

costs 

Strength/Wt 

Ult.load 

1(0) 

^DTRIM 

Cn 

%1NG 


MAINTENANCE 
BLOCK HOUR 

%S 

g’s 

Conventional 

0.88 

0.0007 

0.0070 

0.35 

$0 

86,800 

3.75 

Supercritical 

.92 

.0007 


.35 

0 

86,800 

3.75 

Active controls 

.92 

.0000 

.0070 

.21 

10 

86,800 

3.0 


Class of Airplane Studied: 

Payload - 300 passengers, max 
Range — 3750 n. mi. 

Cruise Mach number - allowed to vary. M 0.84 
Design criteria - ROI or SMRG 
Payload factor — 55% 
































• Rtducad static stability - less trim drag 

• Gust load alleviation - less structural weight 

• Maneuver load alleviation -* less structural weight 

• Ride qualities - passenger comfort, longer fatigue lift 

• Flutter suppression- less-structural weight, longer fatigue life 

• Envelope limiting - flight safety 

• Decoupled control- improved handling qualities 

Figure 2.- Active controls. 


Figure 1.- The job of Integrating pro- 
pulsion, aerodynamics, structures, 
and controls in a conceptual design. 


>1^ • Reduced static stability 

Reduced drag-^less fuel 



Reduced bending moment ♦less weight 



• Gust load alleviation 
Reduced load ♦less weight 



• Flutter suppression 
Increased flutter velocity-^less weight 


Objective 

"The overoU objective of the active control technology 
progrom is to identify, develop, ond validate the controls 
technology required for the integration of active controls 
in future subsonic civil aircraft." 



Figure 4.- Objective of the active con- 
trol technology program. 


Figure 3.- Active control concepts. 
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Return on investment 



= C* = ARG max (ROD 
c 

f(c) = 0 
g(c) <0 


Figure 5.- Overall active control air- Figure 6.- Design criterion, 

craft program. 




d operating cost 


A operoting cost 


> Aweight ^ A initial cost ^ A operating cost 
"" (5657 lb) ■*' ($ lOM) ($l26/hr) 

Aweight cost = Aweight, structure Aweight, system 
Aintioi cost = Aintiol cost, structure A intiol cost, system 

A operating cost = A maintenance cost + A operating cost, liability 
(MTBF, pilot rating) 


Figure 7.- Linearized design criterion. 



Figure 8.- Active control analysis 
package . 



Total seat miles per gallon 


Figure 9.- Return on investment and total seat miles per gallon for various 

designs. 
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Figure 10.- Dynamic pressure locus 
using state space aeroelastic 
model. 


I 


Maneuver load control 

Feedforward, minimize primarily wing weight 
by reducing wing bending moment due to 2.5 g 
maneuver 

Gust load alleviation, 
ride control 

Measure gust, feedforward, desensitize the 
aircraft response through feedback designed 
using reduced state, quadratic cost algorithm 

Reduced static stability 

Optimal reduced state feedback using qua- 
dratic cost function based on pilot ratings 

Flutter suppression 
structural mode damping 

Include aeroelastic modes unsteady 
aerodynamics in models used for control 
synthesis to assure stability 

All systems 

Consider various levels of complexity and at 
several flight and loading conditions. Use 
linearized design criterion to assess candi- 
date system configurations 


Figure 11.- Approach taken in synthesizing active control laws. 
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STATE VARIABLES FOR LONGITUDINAL, SYMMETRICAL MODES ; 


No. of 
states 


Rigid body modes (a, q, 9, V) 4 
Aeroelastic modes (9 second order modes) 18 
Unsteady flow (+2 each mode, control) 26 
Actuator dynamics 8 
Sensor dynamics 10 
Filters, compensators 8 

TOTAL 74 


If all modes are coupled because of engine momen- 
tum or malfunctioning actuator, the total states num- 
ber is about 150. 

Conclusion: 

Need reduced state, insensitive control, "synthesis 
techniques, and algorithms. 


Figure 12.- The problem of too many state variables. 



r,sec 

Figure 14.- Comparison of pilot opin- 
ion boundaries. 
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NASA AMES ACTIVE CONTROL AIRCRAFT FLIGHT EXPERIMENTS (ACA) PROGRAM 
George Meyer and William R. Wehrend 
Ames Research Center, NASA 


The objectives of the Ames ACA program are to develop the active control 
technology (ACT) for short-haul aircraft, to evaluate existing methods, to 
develop new techniques, and to demonstrate the readiness of the technology in 
operational environment. Two concepts are basic to ACT: integrated aircraft 

design and integrated flight-control-system design. These two concepts form 
the basis of the Ames program. 

Studies have shown that significant reductions in induced drag and weight. 
Improvement in passenger comfort, and reduction of flight hazards can be 
achieved with ACT. These benefits are possible due to (a) a reduction in the 
size of stabilizing surfaces, with stability provided by dynamically control- 
ling movable surfaces (reduced static stability design) rather than statically 
with larger fixed surfaces as in conventional design; (b) reductions in struc- 
tural strength requirements by applying maneuver load and gust load allevia- 
tion and flutter mode control; (c) improvement of ride by a ride quality 
control system; and (d) reduction in the occurrence of inadvertent flight 
hazard through automatic limitation of flight condition. The benefits are 
maximized when ACT is factored in the design and integrated with aerodynamics, 
structures, and propulsion early in the design cycle of the aircraft. But, 
before such an ACT-conf igured aircraft can be designed with confidence, 
existing mathematical modeling techniques must be evaluated. In comparison 
with the conventional flight-control-system design, the design of such ACT 
systems as flutter mode control and reduced static stability requires signif- 
icantly more accurate mathematical models of the process to be controlled. 

Of particular concern are the models of unsteady aerodynamics of the basic 
aircraft, control surfaces, and sensors in the flow field; models of flexible 
structures of the airframe, control surfaces, effects of flexibility on sensors 
and actuator linkages, and aeroelastic coupling; and the models of propulsion 
systems, including engine power dynamics and control, and power-aerodynamics 
coupling, the latter being particularly significant in powered lift configura- 
tions. Therefore, a part of the Ames program is directed at the evaluation of 
the complete aircraft modeling methodology. 

The process resulting in an optimum ACT-conf igured design is expected 
to require many iterations on the aircraft geometry, including size and 
location of control surfaces. At each such iteration, the effects of the 
control system must be determined, thus requiring a mathematical model of 
the controlled process. Since the availability of efficient algorithms 
for transforming aircraft configurations into mathematical models for control- 
system design purposes is implicit in the ACT design approach, particular 
attention is given in the Ames program to digital computation methods based 
on potential flow theory (ref. 1). Experiments are being planned to evaluate 
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such methods by comparison with flight-test data. The methodology and algo- 
rithms of system identification are expected to play a prominent role in 
extracting model parameters from flight- test data. 

The second basic concept of Active Control Technology is that maximum 
benefit is obtained from automatics when all functions are blended into a 
single, fully integrated flight-control system. A good working hypothesis is 
that the next generation of flight-control systems will provide four sets of 
functions: (1) ACT modes, including gust load alleviation, maneuver load 

control, flutter mode control, and ride quality control, all of which improve 
aircraft performance without adversely affecting the pilot handling qualities; 
(2) stability and control augmentaion system (SCAS), a set that provides an 
interface between the pilot and the basic aircraft dynamics, such as the 
stabilization of an aircraft with reduced static stability; (3) full-flight 
envelope autopilot, a set whose elements provide the pilot with a variable 
degree of automation, from SCAS (requiring maximum pilot participation) to 
completely automatic four-dimensional trajectory tracking, automatic envelope 
limiting, and configuration management; and (4) automatic fault regulator, 
which provides automatic fault detection, fault identification, and system 
reconfiguration to bypass the fault. 

Before such an integrated flight-control system can be implemented with 
confidence, several system-theoretic problems of varying degrees of difficulty 
must be solved. Powerful concepts (ref. 2) and computerized algorithms exist 
for the design of linear systems and can often be applied to the design of ACT 
modes and SCAS, which are primarily perturbation regulators so that linearity 
and even time invariance are reasonably valid. However, as already noted, 
there may be significant uncertainties in the actual plant parameters and 
plant structure. Methods for the reduction of input-output sensitivity to 
perturbations in the plant are needed. Linear methods are also helpful for 
the design of autopilots, but to a much lesser degree. Increased trajectory 
tracking accuracy requirements of projected STOL and VTOL operations, coupled 
with inherently nonlinear force and moment generation processes in powered- 
lift aircraft, make it very difficult to cover the operational envelope of 
such aircraft with a set of linear models and control laws linked by a gain 
schedule. Alternate methods for coping with the nonlinearities must be 
developed. 

The wide range of lift and drag coefficients required in short-haul 
operation is achieved by in-flight modification of the aircraft configuration 
(ref. 3). Inside the flight envelope there are, normally. Infinitely many 
combinations of controls which result in identical lift and drag coefficients 
— that is, given values of lift and drag define a surface rather than a 
single point in the control space. The remaining freedom can be used to 
optimize system performance and increase both safety margin and aircraft 
maneuverability. The methodology for automatic configuration optimization and 
envelope limiting and for the integration within the autopilot of ACT modes 
and SCAS must be developed. A part of the Ames program is directed at the 
development and evaluation of such a methodology. 
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Normal operation of all system components was tacitly assumed in the 
discussion thus far. When the possibility of component failures is introduced 
into the synthesis, the available synthesis methodology loses most of its 
power. There is a pressing need for major advances in this area, particularly 
in the development of a theoretical foundation for the design of fault regula- 
tors, which detect and identify failures and reconfigure the control logic 
into the next safest mode of operation. The need for fault regulators is 
evident in the case of advanced flight-control systems with high authority, 
flight critical functions. However, it would seem that the majority of control 
systems in the future, regardless of the particular field of application, will 
be controlling with high, if not complete, authority complex, critical systems 
and therefore will have to regulate faults automatically. Thus, advances 
to be made in flight-control systems will very likely have wide-ranging 
applications . 

A part of the Ames program is directed at the development and evaluation 
of the fault regulation methodology.' The scope is limited to (1) synthesis of 
the reconfiguration logic, given a complete description of every failure from 
a given set; (2) sensor subsystems that provide, through hardware redundancy 
and functional redundancy, state estimates and their accuracy including 
extreme failures; and (3) actuator subsystems that provide, through hardware 
redundancy, estimates of command execution accuracy. Other topics, such as 
hardware failures in the flight computer, are excluded from the program. 

All three parts of the Ames program, namely, modeling methodology. 
Integrated flight-control-system design assuming no component failures, 
and fault regulation methodology will be developed and evaluated by generaliza- 
tion from particular applications to specific research aircraft. In the first 
phase of the program, research will be focused on the DHC-6 (Twin Otter) 
aircraft — an example of a low wing loading configuration with conventional 
lift. The aircraft has been modified to include a set of spoilers for direct 
lift, drag, and roll control. In addition, the aircraft has been equipped 
with a STOLAND system, including a flight computer with access to a large 
set of sensors and actuators. In the second phase, the research will be 
focused on powered-lift configurations similarly equipped with functionally 
powerful hardware. Candidates for that phase are XV-15 (tilt-rotor) research 
aircraft and possibly either YC-14 or YC-15 (advanced medium STOL transport) . 

The first major objective of the program is to formulate a complete 
overall logical structure for automatic flight-control systems that can 
(1) admit any combination of ACT modes; (2) provide accurate, automatic cou- 
pling to air traffic control, which may select any trajectory from a large 
set of complex trajectories; (3) provide a spectrum of modes of pilot- 
automatic interaction; (4) accept a variable sensor set of variable accuracy 
and a variable actuator set; (5) accept a highly nonlinear plant with function- 
ally redundant controls; and (6) provide a framework for the design of fault 
regulators. The logical structure being developed at Ames is described 
briefly here. (A more detailed description is given in ref. 4.) 

The structure of the control logic is hierarchical in the sense that 
there is a nesting of subsystems. At each level in the hierarchy, relatively 
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simple commands are accepted from above and transformed Into more detailed 
commands downward. The command Is based on a relatively simple Input-output 
model of lower levels, and the levels have a degree of autonomy. Figure 1 
shows the control logic as viewed from the level of the trajectory control 
system. At the top of the hierarchy Is the air traffic control (suitably 
generalized to Include the pilot) which requests a trajectory that may or 
may not be executable from the existing state of the aircraft. The trajectory 
command generator transforms this request Into an executable open-loop 
acceleration command and a corresponding four-dlmenslonal trajectoiry that may, 
depending on the original request, have Ignorable parameters. The accelera- 
tion command Is sent downward In the hierarchy to the trajectory acceleration 
controller, which Is expected to execute that command In accordance with a 
relatively simple Input-output relation and error bounds. Errors arising from 
wind turbulence, uncertainties In plant parameters. Intentional oversimplifi- 
cation of the dynamics, etc., are controlled by means of the trajectory 
perturbation controller In which estimated aircraft position, velocity, and 
acceleration are compared with the command, and a corrective acceleration 
generated and added to the open-loop command, thereby closing the loop around 
the errors. 

Figure 2 shows the structure of the trajectory acceleration controller 
for the case In which lift Is controlled by angle of attack and side force Is 
controlled by the roll angle, so that aircraft attitude becomes an Indirect 
control variable. The closed-loop acceleration command enters the force 
trlmmap, an algebraic (static) representation of the total (aerodynamic and 
propulsive) force generation process. At this level, configuration management 
and envelope limiting take place. The output Is the attitude and remaining 
controls which. In the absence of errors, would trim the aircraft to the 
required acceleration. The trim attitude, possibly time-varying. Is sent to 
the attitude control system, which Is similar to the trajectory control system. 
The trim attitude Is transformed In the rotation command generator into an 
executable open-loop angular acceleration command and a corresponding angular 
velocity and attitude time histories. The angular acceleration command is 
sent down the hierarchy to the angular acceleration controller, which is 
expected to execute the command with a relatively simple input-output relation 
and error bounds. Attitude errors are controlled by means of the attitude 
perturbation controller in which estimates of aircraft attitude angular veloc- 
ity and angular acceleration are compared with the command, and a corrective 
angular acceleration is added to the command, thereby closing the loop around 
the errors. In figure 2, the angular acceleration controller is represented 
by the moment trlmmap. The control surface servos, at the bottom of the 
hierarchy, are not shown. 

The proposed logical structure has several advantages, among which are 
the following. It is applicable to a large class of aircraft. The particu- 
lars of the aircraft affect primarily the force and moment trimmaps. The 
design approach is nearly algorithmic. Performance, sensor accuracy, and 
actuator accuracy enter as Independent variables that may be varied over a 
wide range. There is an effective tradeoff between performance and flight 
computer requirements. Because the control logic is hierarchical, problems 
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associated with the design of fault regulators are considerably simplified. 
A flight-test evaluation of the control logic Is expected to occur In the 
summer of 1976. 
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Figure 1.- Trajectory control system. 



Flap, throttle, DLC 


Figure 2.- Trajectory acceleration controller. 
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SUMM/^RY 


This paper describes an adaptive flight-control-system design for NASA's 
F-8 digital fly-by-wire research aircraft. This design implements an explicit 
parallel maximum likelihood identification algorithm to estimate key aircraft 
parameters. The estimates are used to compute gains in simplified quadratic- 
optimal command augmentation control laws. Design details for the control 
laws and identifier are presented, and performance evaluation results from 
NASA Langley's F-8 Simulator are summarized. 


SECTION I - INTRODUCTION 


NASA is presently conducting research in digital fly-by-wire flight- 
control technology. In the first phase of this research, an F-8C test air- 
craft was modified to use an onboard Apollo guidance computer for flight- 
control calculations. The computer was connected directly to the aircraft's 
electrohydraulic actuation systems, with no mechanical linkages remaining 
between pilot stick and control surfaces. This configuration was flight 
tested to demonstrate the practicality of digital fly-by-wire control in an 
actual test vehicle (ref. 1). 

For the second phase of the research program, the test aircraft is being 
outfitted with triply redundant AP-101 computers, which are again connected 
directly to (improved) electrohydraulic actuation systems. This new config- 
uration serves as a test bed for studies and demonstrations of redundancy con- 
cepts to enhance reliability and advanced control laws to improve the perfor- 
mance and/or overall effectiveness of the aircraft. For control law studies, 
in particular, the new computer's speed, memory size, and floating-point 
capability removes most of the complexity constraints imposed by more typical 
flight-control machines. 

To date, two sets of control laws have been scheduled for flight experi- 
mentation in the new F-8 configuration — a "CCV package" and an "adaptive 
package." The first package is described in reference 2. It provides basic 
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command augmentation functions, outer loops, and control inodes for various 
control-configured- vehicle (CCV) concepts applicable to fighter aircraft. The 
second control law package is described here with further details in refer- 
ence 3. It consists of the same command augmentation functions found in the 
CCV package, but adds explicit on-line maximum likelihood identification capa- 
bility for adaptive control. In addition to these packages, research has also 
proceeded on several other candidate adaptive algorithms for the F-8 
(refs. 4-6) and on sensor fault detection and isolation algorithms (refs. 7 
and 8) . Some of these will reach flight experimentation later in the program. 

The purpose of this paper is to describe the contents of the Adaptive 
Control Law package. We begin the description (section 2) with a discussion 
of general design objectives and ground rules. We then introduce the overall 
adaptive system structure and discuss the design of its major components — the 
control law (section 3.2) and the explicit identifier (section 3.3), followed 
by simulator performance evaluations in section 4 and by a summary of remain- 
ing issues and plans in section 5. Of necessity, many of the modeling, 
design, and evaluation details caiinot be covered here. For these details, the 
interested reader is referred to references 2 and 3. 


SECTION II - OBJECTIVES AND GROUND RULES 


NASA's primary reasons for studying adaptive control laws for the F-8 
aircraft are to identify potential performance and system effectiveness bene- 
fits offered by modem adaptation/identification concepts and to demonstrate 
these in an actual digital flight- control environment. Hence, the aim of the 
design effort was not to develop new theoretical procedures or algorithms but 
to turn existing concepts into flightworthy control laws for the specific test 
aircraft. Of course, the concepts and design processes should be general 
enough to apply to other aircraft as well. This is important because the F-8C 
itself is not a state-of-the-art fighter, nor is it difficult to control with 
standard nonadaptive techniques. Other more specific groundrules imposed on 
the design include the following. 


Inputs 

The adaptive system must operate in the presence of normal pilot inputs 
and also when such inputs are absent. Any test signals required for the 
latter case must be small enough not to interfere with the aircraft's mission. 
This generally means that test input normal accelerations, as sensed at the 
pilot station, should be below 0.02-0.03 g rms, and lateral acceleration 
should be even lower in the range of 0.01 g. This ground rule establishes a 
crucial distinction between identifiers designed for operational adaptive con- 
trols and those designed for postflight-test data-processing applications 
(refs. 9 and 10). In the latter case, test inputs are deliberately large and 
often optimized for identification accuracy (refs. 11 and 12). For opera- 
tional adaptive controls, the emphasis must be to make these inputs small and 
hopefully unnoticeable. 
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Sensors 


The system was constrained to operate with the aircraft’s existing sur- 
face (servo) position transducers and inertial sensors (rate gyros and accel- 
erometers) , but without attitude measurements and without air data measure- 
ments (dynamic pressure, velocity, angle of attack). These limitations are 
imposed by the sensor complement available on the test aircraft and, in the 
case of air data, by the philosophical bent of the authors. We believe that 
removal of flight-critical air data measurements is one of the most tangible 
benefits adaptive controls can offer for aircraft (like the T-8) whose basic 
performance requirements are readily satisfied with air-data-scheduled control 
laws . ^ 


Computer Capacity and Sample Rate 

Since control law calculations typically consume only a small fraction of 
the total computational load in a flight computer (the bulk is I/O, self test, 
mode and redundancy control), the adaptive law’s goals are restricted to 
25 percent of the available frame time of the AP-101 and to a .Vreasonable” 
allocation of memory. The sample rate of the computer was prespecified to be 
50 Hz. This gives 5 msec real time per sample in which to complete all con- 
trol and identification calculations. The sample rate is also high enough to 
produce no substantial differences between direct digital design (discrete- 
time control laws designed for discrete-time models) and continuous time 
design with after-the-fact discretization. 


Control Surfaces and Actuators 

The standard elevator, aileron, and rudder surfaces were available for 
adaptive control. Leading-edge and trailing-edge flaps were assumed to remain 
imdeflected or to follow their open-loop scheduled minimum drag positions. 
Actuation systems for these surfaces were represented by models incorporating 
projected characteristics of the improved systems being installed on the test 
aircraft. 


SECTION III - ADAPTIVE SYSTEM DESIGN 


The overall structure chosen for the F-8’s adaptive control system is 
based on explicit separation of identification and control (fig. 1). A stan- 
dard gain- scheduled control law was designed for the aircraft’s complete 
flight envelope and the scheduling functions (gain adjustment rules) are 


^In flight-critical digital fly-by-wire control systems, reliability 
requirements call for triply redundant or even quadredundant air data sensors. 
Aside from the cost and complication of these multiple sensors, suitable probe 
locations impose difficult compromises between data quality, channel tracking, 
and dispersion to avoid common hazards. 


29 


supplied from a separate aircraft identifier. While such a priori separation 
is known to be theoretically imperfect in the dual-control sense (ref. 13), it 
is ideally suited, both historically and in terms of practical features of the 
solution, to the flight- control problem. This will become evident as we 
describe aircraft models and features of the separate controller and identi- 
fier designs. 


Aircraft Models 


Mathematical models for the F- 8 C were developed from general six-degree- 
of-freedom nonlinear equations of motion together with geometric data, mass 
properties, and aerodynamic data tables (ref. 14). These equations were 
linearized numerically at 25 flight conditions throughout the operating enve- 
lope of the aircraft. Each flight condition is defined by the nominal alti- 
tude, Mach number, weight, geometric configuration, and load factor which 
characterize the models obtained by the linearization, that is. 


X = F(c^)x + Gj(c ^)6 + G 2 (c^)w + GgCc^) 
y = H(c^)x + D(c^) 6 , i=l,2,...,25 


( 1 ) 


where x denotes 12 aircraft states, 6 denotes 3 control surface deflec- 
tions, w denotes wind gust disturbances, and y denotes assorted outputs we 
wish to measure and/or control. The matrices F, G 2 , G 3 , H, and D can be 

treated as continuous functions of the slowly changing nominals jc, with 
values known at discrete points. 


Equation (1) is not usually used directly for control design. It is 
first partitioned into longitudinal and lateral-directional degrees of free- 
dom, and each partition is separated into dynamic modes that dominate the fre- 
quency range of interest in the design task (ref. 15). The resulting reduced 
models are then augmented with actuator, sensor, gust, and flexure character- 
istics as necessary to make up the complete design model. 


For the F- 8 C adaptive system, the control objective was to provide basic 
stability and command augmentation functions. The reduced models that suffice 
for this objective consist of second-order short-period dynamics plus a first- 
order power actuator for the longitudinal axes. A fourth-order dutch roll, 
roll subsidence, and spiral mode model suffices for the lateral-directional 
axes. The longitudinal model is summarized in figure 2 and table I, and the 
lateral-directional model can be found in reference 2 . 


Figure 2 and table I provide model equations and approximate functional 
descriptions for the equation coefficients, respectively. The latter func- 
tions were derived by plotting numerical values from the 25 flight conditions 
against two dominant nominal flight condition parameters, d 3 mamic pressure 
(q) , and Mach number. The trends in these plots were approximated by simple 
functions, and the scatter was represented by additional small-perturbation 
parameters with bounded uncertainties. Hence all coefficients have the form 
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= f(q, Mach, c^) 


( 2 ) 


or the form 


p± = Mach, c^) (3) 

where equation (3) is obtained by substituting the very good linear relation- 
ship between surface effectiveness (M^q) and q into equation (2) . The point 
of including these model data here is to show how strongly the parameter 
(or equivalently q) dominates the F-8C’s dynamics. Consequently, this same 
parameter also plays a dominant role in the control law and identifier designs 
to follow. 


Control Law Design 

Both the longitudinal and lateral-directional control laws were designed 
with modern LQG techniques (ref. 16) appropriately tempered with established 
classical design requirements. In each case, a quadratic optimization problem 
was formulated and solved at selected flight conditions under assumptions of 
noiseless, full-state, continuous- time feedback. The resulting control gains 
and dynamics were then simplified and approximated as functions of measurable 
nominal- flight condition parameters to provide a standard gain- scheduled con- 
trol law. We describe this design process in more detail below for the longi- 
tudinal control law only. The lateral-directional details are again provided 
in reference 2. 

Longitudinal optimization problem- To gain ready acceptance as flight- 
worthy, the F-8C*s longitudinal control laws were designed to satisfy the 
following list of classical performance specifications: 

1. Command augmentation functions 

a. C* step response 

b. Stick gradient to meet MIL-F-8785 (ref. 17) 

c. Neutral speed stability 

2. Regulator function 

Reasonably damped gust responses (short-period damping ratio > 0.3) 

3. Tolerance for uncertainties 

a. Classical loop gain and phase margins (CM > 6 db, <|) > 30°) 

b. Loop attenuation at high frequencies (< -20 db at 8 Hz) 

These requirements are obviously not in the quadratic cost function form 
required by LQG theory. Hence an important step in the design process was to 
reinterpret them in more suitable terms. 
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Consider the command augmentation requirements first. Item (la) in this 
group is a dynamic response specification on C*, a linear combination of 
pitch rate and normal acceleration: 

G* = nj, + = 324 (4) 

In response to step commands from the pilot *s control stick, this variable 
must stay within a specified response envelope (ref. 18) whose center is 
closely approximated by the step response of a second-order linear model with 
0 ) ss 7 rad/sec, ? = 0.9. This requirement was incorporated in the LQG frame- 
work by the common technique of appending an explicit "command model" and 
penalizing errors between it and the aircraft (refs. 19 and 20). Specifica- 
tion item (lb) was then used to set the DC gain (acceleration per unit stick 
force) of the model, and item (Ic) was satisfied by penalizing an appended 
integral of the model- following error as well as the error itself. The latter 
approach is motivated by the fact that "neutral speed stability" is actually 
imposed to provide command insensitivity to trim changes. A pilot should not 
be obligated to readjust trim or to provide compensating stick force inputs as 
the aircraft changes speed (and hence as it changes the trim disturbance terms 
Mq and g/V in fig. 2). A bit of integral control action provides this 
desired insensitivity.^ 

In addition to the command model and integrator appended to satisfy 
items (la) to (Ic) , it was also necessary to append a low-pass filter on the 
normal acceleration signal used to construct C*. This filter provides high- 
frequency attenuation, item (3b), which is required to assure adequate stabil- 
ity margins in the presence of uncertain servo characteristics and unmodelled 
flexure dynamics of the airframe. Loop transfer functions of trial designs 
without such filters tended to drop off too slowly to meet the specification. 
Slow roll-off is, of course, a general property of quadratic-optimal control 
laws, formally recognized but frequently forgotten (ref. 21). 

The complete optimization problem with these appended states is illus- 
trated in figure 3. As shown, the criterion function was taken as a weighted 
sum of four quadratic terms — model- following error (C* - C*^) , integrated 
model-following error /(C* - elevator rate (d/dt)6^, and elevator 

command 6^. Quadratic weights were then selected iteratively until the 
resulting control law satisfied all performance specifications. Representa- 
tive weights and gains are summarized in table II. 

Contvol taw S'impt'tfioat'ton- As indicated earlier, the quadratic optimal 
control law was next simplified and approximated as a function of nominal 
flight condition parameters. While several numerical algorithms have been 
developed to aid in these reductions (refs. 20, 22, and 23), the F-8C*s longi- 
tudinal control law was compact enough to be reduced by standard block diagram 
manipulations, cancellation of nearly equal poles and zeros, and removal of 
low-gain paths. The result is shown in figure 4. It consists of a basic 
proportional-plus- integral C* feedback loop, modified slightly by the 


^Neutral speed stability is desirable during all flight phases except 
power approach. For the latter, the integral action must be switched out. 
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appended lag network on measured acceleration and by a lead-lag network on 
pitch rate. The pilot stick force commands this loop through a feedforward 
filter that consists of the original C* model with altered numerator dynam- 
ics, All parameters in the 'control law ar^ independent of flight condition 
except the loop gain, Gc*> which varies inversely with surface effectiveness 
(or dynamic pressure). It becomes desirable to adjust, other gains only if the 
aircraft is flown with deliberately relaxed static stability.^ Foro such con- 
ditions, the dotted line path in figure 4 must be added to the control law, 
with gain adjusted as a function of Mgo (or q and Mach). 

Numerical values for the gain Gq* are shown in figure 5, where they are 
also compared with the critical gain at which the (linear) loop reaches insta- 
bility. It is clear from these curves that the F-8C is a very tolerant 
aircraft. easily achieves 6 db of margin over the entire flight enve- 

lope, and even a fixed gain system could fly the aircraft adequately at all 
but the lowest dynamic pressure conditions. This fact deserves emphasis 
because it complicates evaluations of an adaptive system designed for this 
aircraft. Total closed- loop performance does not provide a sensitive measure 
of adaptive capability, and other less direct criteria are required to 
evaluate the system. 

Lateral- drive ct'ionat control laws- Although details are left to refer- 
ences 2 and 3, the lateral-directional control design process resulted in 
structures very similar to figure 4. The roll channel in figure 6 is a simple 
roll-rate feedback loop commanded through a first-order model, while the yaw 
channel in the figure combines compensated yaw rate, roll rate, lateral accel- 
eration, and aileron position feedbacks with direct rudder pedal commands. 

Both loops have one gain that changes with flight condition in inverse rela- 
tion to d 3 mamic pressure and hence in direct proportion to gain G^^. 


Identifier Design 

In nonadaptive flight-control implementations, variable gains like those 
in figures 5 and 6 are usually adjusted with external measurements obtained 
from the aircraft's air data system. This approach will be used in the F-8C's 
first flight tests of the CCV Control Law Package planned for mid-1976. As 
already noted, however, air data systems can be troublesome components in the 
overall flight- control mechanization, and it would be very desirable not to 
need them. The identifier design is intended to accomplish this end. Its 
primary function is to provide an adequate surface effectiveness estimate 
(M^o) to substitute for measured q. It also estimates several other quanti- 
ties — pitching moment due to angle of attack (M^^) , true airspeed (V) , and 
total angle of attack (5) . These are sometimes needed to schedule more com- 
plex control laws than those that suffice for the F-8. 


^This is a potential CCV feature achieved by large aft displacements of 
the c.g. Its benefits are reduced trim drag at the expense of basic airframe 
^ stability. In the model coefficients of table I, it means replacing with 

the functions <^ 2^60 (0.92 + Mach < 1 and Mach > 1, respec- 

tively. There are no plans to fly the F-8C with these modifications. 
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Several identification schemes were examined to perform these functions, 
including approximated (second-order) nonlinear filters, Lyapunov- stable model 
trackers, maximum likelihood estimation (MLE) , and self-excited limit cycle 
systems. Trial designs were actually carried out for the latter three 
(ref. 3). The MLE procedure was selected out of these primarily because it 
offers the greatest capability in terms of the numbers of parameters identi- 
fied and their accuracy. Details of the MLE design are discussed below. 


Identic fiaation aQQuraay- Before proceeding with the identifier descrip- 
tion, we present some evaluations of theoretical limits on the accuracy to 
which parameters of the F-8C can be identified under the ground rules imposed 
in section II. These evaluations provide justification for most of the design 
decisions made for the MLE identifier. The accuracy evaluations were per- 
formed for the identification problem illustrated in figure 7, The aircraft 
(as modeled in fig. 1) with a simple pitch rate feedback control loop was 
assumed to be excited by small test signals, wind gusts, and sensor noise. 
Discrete pitch rate, normal acceleration, and surface position observations 
were assumed available to identify unknown dynamic coefficients (as parameter- 
ized in table I), gust level, trim disturbances., and initial conditions. It 
is well known that the achievable identification accuracy for this problem is 
limited by the following lower bound (ref. 24): 



where k = 1,2,..., is the sequence of residuals generated by a Kalman 

filter designed for the true parameter values; k ~ 1,2, . . . , is the 
covariance matrix of that sequence; Pq is the covariance matrix of initial 
parameter uncertainties; and T is the data length. 


The situation in figure 7 was simulated, and sample averages of the 
matrix M were computed for various combinations of data length, test signal, 
gust, and sensor noise conditions. The principal result of these calculations 
is shown in table III, which compared square roots of selected diagonal ele- 
ments of Pq with corresponding elements of M(c^,T). The comparisons^ are 
made for T = 5 and T = 10 sec and correspond to the following environmen- 
tal conditions: 


Flight condition: 
Test signal: 

Gust level: 

Sensor noise: 


20,000 ft, Mach = 0.67, q == 305 Ib/ft^ 

Square wave at short-period frequency, 
0.015 g BMS acceleration 

1 ft/sec la 

Og = 0.015°/sec, ~ 0.02 g 


**Because parameters can change significantly over 10 sec, 
meaningful to look at longer data lengths. 


it is not 
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Table III shows that we can expect only limited identification capability 
under these conditions. Our a priori knowledge improves for only two dynamic 
coefficients, and C 2 (the small perturbation of , and for a^jg, trim, 

and initial conditions (not shown) . Accuracies are lifted to approximately 
10 percent for both and Fortunately, these are adequate in most 

gain-scheduling applications. Using similar analyses for other test condi- 
tions, the following additional conclusions were established: 

Accuracy limits: Mgo and errors are proportional to true parameter 

values* Accuracies fall into the same 10-percent range for all flight 
conditions. 

Reduced parameter identification: Identifiers desigped to find only a 

few parameters (as few as trim, and ICs) do almost as well as identi- 

fiers designed to find all parameters, even in the presence of errors in the 
unrecognized parameters.- However, trim disturbances and initial angle of 
attack cannot be deleted without inducing major errors in Mgo* 

Test signals: Randomized test signals are superior to cyclic ones for 

reduced parameter identification. 

Lateral-directional identification: Identifiers for the lateral- 

directional axes reduce a priori uncertainties for only two parameters, q and 
Ng (yawing moment due to sideslip) . Accuracies are comparable to longitudinal 
identification. 


MLE identifier struotvcre- In view of the above identifiability limita- 
tions for the F- 8 C aircraft,^ it was decided to utilize a longitudinal-axis 
identifier only with a reduced parameter set of three components — M(So> 
and C 3 (small perturbation on velocity).^ It was also decided to avoid the 
iterative nature of conventional MLE algorithms (refs. 9 and 10) by adopting 
a parallel approach to likelihood minimization. This approach is illustrated 
in figure 8 . It consists of a bank of Kalman filters, each generating suffi- 
cient statistics for the conditional probability distribution of the measure- 
ments given a different value for the unknowns, that is, 


T/At 

P (71,72. •••, yT/At|-i^ “ 1? N[yj^(c^)., B^(c^) ] \ 

1 = 1,2, ..., N ; 


( 6)7 


is important to recognize that the limitations cited above are F- 8 C 
specific and are caused primarily by the low test signal ground rules Imposed 
in section II. 

^Though only weakly identifiable, C 3 was carried along for flight 
experiments with larger test signals. 

^N[x,B] denotes the usual multivariate normal distribution with mean x 
and covariance matrix B. 
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where and are predicted measurements and residual covariances, 

respectively,^ from the ith Kalman filter designed for ^ = c^. Likelihood 
functions (L = -Jin p) are then computed for each filter channel, the minimum 
likelihood channel is selected, and a single (approximate) Newton-Raphson step 
is taken from there to predict the parameter value, that is. 


T/At 

U^.T) - 2: 

k=0 




^ + £n det B, (c, ) 

V<£i> . 

i = 1 , 2 , ... , N 


1 * = 


Arg|min L(Cj|^,T)j. 


(7) 

( 8 ) 


1 = H (c.*,T) (9) 

where M(^,T) is the approximate second-par tials matrix defined in equa- 
tion ( 5 ) . This parallel minimization approach is a modified form of several 
procedures published in the literature (refs. 25 - 27 ). Its primary advantages 
for onboard use are recursiveness and fixed structure. The filters and sensi- 
tivity calculations implicit in equation ( 9 ) process measurement samples as 
they appear and use a fixed set of repetitive program instructions. No data 
storage and conditional iterative reprocessing are required. They also remain 
fixed in parameter space, requiring no onboard calculation of new plant 
models, Kalman filter gains, and their associated sensitivities. These are 
all desirable features for real-time onboard computer code. 


Design specifics- A specific implementation of figure 8 Involves the fol- 
lowing design choices and issues: 

• Identification models 


• Channel selection 


• Kalman filter design parameters 

• Adjustments for practical situations not handled by the basic 

theoretical approach 

These items are discussed below for the F-8C’s specific design. 

Identification modets- Each Kalman filter channel in the F-8C design uses 
a discretized version of the basic aircraft model in figure 1 with the actu- 
ator state deleted (6^ is assumed measurable) and with the two trim distur- 
bances appended as Brownian motion “trim states." These trim states eliminate 
the need (identified above) to treat the disturbances as explicit unknown 
parameters. The model’s dynamic coefficients are parameterized as shown in 
table I, but c^, c^ , and c^ are assumed to be zero. Only 0.^9 C3, and Mgo 
are recognized as unknowns. 
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channel seteation- The three unknowns generate a parameter space illus- 
trated in figure 9. Two axes are defined by M^q and (or equivalently M^) 
and the third axis (not shown) is defined by c^. The problem of channel 
selection is to choose both the number and the location of points in this 
space at which to operate filters. We obviously want as few channels as pos- 
sible, yet they must be close enough together to be able to interpolate with 
one Newton-Raphson step (eq. (9)). A bit of experimentation showed that five 
channels suffice for the F-8C (located as shown in the figure) . Four cover 
the subsonic range and are logarithmically distributed along the M^q axis, 
with both c^ and c^ equal to zero. The logarithmic arrangement is motivated 
by our earlier finding that expected identification errors are proportional to 
true parameter values. The fifth channel covers supersonic flight regimes. 
Each channel interpolates over an approximate ±50-percent range about its 
nominal value. 

Katmcm fittev design’- Kalman filters and sensitivity filters for each 
channel were designed under steady-state assumptions, using the following 
fixed values for gust intensity, sensor noise, and trim disturbance growth 
rates : 
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In addition, the surface position measurement was assumed to be corrupted by 
discrete additive white noise with 0.04° rms intensity. These numbers can be 
considered roughly representative of the F-8C’s environment. 

Adjustments- The basic MLE design required suprisingly few after-the-fact 
” fixes” to make it work in situations not wholly consistent with the theoreti- 
cal maximum likelihood problem formulation. Two such adjustments are worth 
mentioning here. The first deals with the fact that the aircraft’s parameters 
are not really constant, as assumed in the design so far. This means that we 
cannot accumulate likelihood functions and their sensitivities indefinitely, 
as implied by equations (5) to (9) . Rather we must "forget” past data by some 
expedient such as the high-pass operations used in figure 8. The high-pass 
approach was selected primarily because it preserves the overall recursive 
nature of the algorithm. Time constants (t) were set at 5 sec, an experimen- 
tal compromise between systematic tracking errors occurring for t too large 
and random errors or false minimums occurring for t too small. 

The second adjustment deals with time- varying noise statistics. These 
can cause substantial estimation errors in identifiers designed for fixed 
statistics. Consider, for example, a case where gust, sensor noise, and trim 
disturbances are actually very small compared to the nominal design values. 
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Then the likelihood function In equation (7) will be dominated by the 
In det B(^) terms. These depend on a priori channel data only, not on incom- 
ing measurements. For the F-8C, they are minimized at channel 1, Mgg = -2.34, 
regardless of the true M§q value. Hence they can produce large estimation 
errors. . This potential problem was alleviated by scaling the An det B terms 
(and their gradients) with the constant 


T/At 

0^ = min 

^ k=0 


^k - 




(2T/At) 


( 10 )® 


which can be formally derived by considering a to be an unknown common scale ^ 
factor on all noise statistics to be identified as part of the MLE procedure 
(ref. 3). 


The F-8C identifier incorporates several other less important adjustments 
not covered here. These include interpolation of state estimates between 
channels, automatic initialization and data transfer when channel changes 
occur, significance tests on channel selections, and high-frequency roll-offs 
on sensitivity accumulation to prevent sharp estimation transients. These are 
covered in reference 3. Reference 3 also provides computer sizing estimates 
for the complete identifier. These call for approximately 2200 words of 
memory and 6 msec of execution time per data sample. 


SECTION IV - ADAPTIVE SYSTEM PERFORMANCE 


The control laws and identification algorithms described in the last sec- 
tion are scheduled for flight evaluation during 1976-77. In preparation for 
these tests, they have been mechanized and evaluated on the F-8 simulator at 
Langley Research Center and are also scheduled for evaluation on the simulator 
at Dryden Flight Research Center. Only Langley’s evaluations are available to 
date. Key results are described below. 

Langley’s F-8 simulator is an extensive six-degree-of-f reedom nonlinear 
digital simulation program. It includes complete aircraft dynamics, actu- 
ators, servos, flexibility, sensor models, and real-time interfaces with an 
F-8 iron-bird cockpit and with standard simulation consoles (ref. 14). This 
facility was used to evaluate four ’’measures of goodness” of the adaptive 
system: 


• Identification accuracy in steady flight 

• Convergence characteristics 

• Tracking characteristics for standard flight transition 


®As in equations (5) to (9) , the accumulation in equation (10) and the 
growing divisor (2T/At) are actually high-passed. 
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• Responses to major maneuvers and configuration changes 

These criteria deliberately concentrate on identifier performance. Overall 
closed- loop performance, as mentioned earlier, provides a less sensitive mea- 
sure of adaptive capability. 


Accuracy at Fixed Flight Condition 


Identification accuracy (and thus control gain accuracy) was verified by 
subjecting the closed- loop adaptive system to a standard sequence of test con- 
ditions while in trimmed flight at several fixed flight conditions. The test 
sequence consisted of a period of quiet, followed by a 20-ft/sec^ pilot 
doublet C* command, followed by a period of atmospheric turbulence, and then 
again by a 20-ft/sec^ doublet. This entire sequence was repeated with and 
without sensor noise and always includes a small random C* command as a test 
signal. Accuracy results for these tests are summarized in table IV and 
verify the theoretically predicted identifier performance. Typical transient 
responses are given in figure 10. 


Convergence Characteristics 

To verify convergence properties, the algorithm's mln-select was initial- 
ized at a preselected channel and required to converge to a flight condition 
covered by another channel. This was done for several combinations of chan- 
nels and input conditions under normal closed- loop operation. Convergence 
times (to within ±20 percent of final value) were less than 1 sec for 

all cases. Typical examples are shown in figure 11. 


Tracking 

The system's ability to track changing aircraft parameters was tested by 
flying through standard flight transitions. One such' transition is shown in 
figure 12. It consists of maximum level accelerating flight starting at low 
dynamic pressure, h = 20,000 ft, Mach 0.40, q = 109 psf, and terminating just 
above Mach 1.0. To achieve repeatability, the transition was flown without 
pilot inputs. We simply applied 100-percent military power plus full after- 
burner and held pilot- commanded C* and lateral-directional commands at zero. 
The resulting transients in figure 12 verify smooth tracking behavior and 
proper^ channel changes as dynamic pressure builds up. Note, in particular, 
that Mqj is discontinuous as the identifier decides that the aircraft has 
gone supersonic. This jump is consistent with the parameterization in 
table I. Figure 12 includes sensor noise and 6-ft/sec RMS vertical gusts. 


Major Maneuvers 

The closed- loop adaptive system was also tested under large maneuver con- 
ditions and configuration changes. These Included various rolling maneuvers, 
high-g turns, step angle-of-attack and sideslip disturbances, and speed 
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brake-, gear-, and wing- transition transients. These were largely flown under 
pilot control from the F-8 iron-bird cockpit. They were intended to explore 
qualitative properties of the concept under large signal condition and to 
assure that no drastic upsets occur during normal flight operations. Tran- 
sient traces are documented in reference 3. 


SECTION V - CONCLUSIONS AND REMAINING ISSUES 


The nonlinear simulation evaluations in section IV verify that the paral- 
lel channel maximum- likelihood identifier works well throughout the flight 
envelope and provides adequate gain-scheduling signals for the control law. 
This is ample motivation to proceed to flight test. Like most system designs, 
however, the complete adaptive controller is not ideal and it is important to 
highlight remaining Issues and limitations. These Include tracking errors, 
compromises due to fixed filter design statistics, and test signal 
requirements . 

The tracking error limitation is evident in figure 12. Because of our 
5-sec (effective) likelihood accumulation time, parameter estimates tend to 
lag in the presence of rapid parameter changes. These lags are responsible 
for the largest errors seen in the simulation trials. Errors as high as 
30 percent are evident in the Aq trace in figure 12, and 50 to 70 percent 
errors have been observed during configuration changes and more dramatic 
flight transitions (e.g. , high-g pulldowns). Research is presently under way 
to provide some means of dealing with these time variations. 

The second limitation is evident in most of the accuracy runs. Parameter 
estimates shift somewhat between quiet or sensor-noise-only conditions and 
turbulence conditions. This appears to be caused by inconsistencies between 
true environmental statistics and the statistics used to design Kalman filter 
channels. Some on-line adjustment of these statistics may be desirable. 


Test signal requirements pose a potential limitation not quantifiable at 
this point. Required levels are determined primarily by small nonllnearltles 
and actual noise characteristics in the flight-control hardware, while accept- 
able levels are determined by pilot considerations. In the simulations at 
Langley Research Center, a small random C* command signal (white noise 
passed through the second-order filter s/(s^ + 15s + 36)) was found adequate. 
Its RMS magnitude was adjusted to C* - 4 ft/sec^, which produced the follow- 
ing approximate accelerations at the pilot station: 


Flight condition Test signal acceleration 


Low q (h = 20,000, M = 0.4) 
Medium q (h = 20,000, M = 0.67) 
High q (h = 3000, M = 0.8) 
Supersonic (h = 40,000, M = 1.2) 


0.01 g RMS 
.025 g RMS 
.04 g RMS 
.02 g RMS 


These levels can be lowered somewhat with tolerable accuracy degradations. 
They cannot be used with much certainty, however, to predict the levels 
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needed for the actual aircraft. For these requirements, we must wait until 
flight. 
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TABLE I.- F -8 LONGITUDINAL MODEL PARAMETERIZATION 

Small 

perturbation 
parameter 


^2 


^3 
^5 

Cy I 2.0 


^This is "unflexed" surface effectiveness. Actual surface effectiveness 

taking quasi-static structural bending into account is given by 

M- - (1 + 0.016M- + 0.0002m| )M. . 

0 6 o So So 


Perturbation 

parameter 

uncertainty 

0.065 

0.135 

31.5 

5.0 

15.0 

0.75 


Equation coefficient 



= 

-0.23 + 0.028 MgQ + cj 

Mach < 1 


- 

-0.23 + 0.010 MgQ + Cj 

Mach > 1 

Ma 


(0.61 + C 2 >M^^ 

Mach < 1 


= 

(1.53 + C 2 )M^^ 

Mach > 1 

V 

= 

(200 + C 3 ) - 

Mach < 1 


ss 

(260 + C 3 ) - 

Mach > 1 

ZaV 

= 

(53 + c,)M^^ 

All Mach 


= 


All Mach 

" 6 ^ 

= 

(7.7 + cg) 

All Mach 


q = (2 + All Mach 
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TABLE II.- LONGITUDINAL WEIGHTS AND GAINS 


Flight condition 
Altitude (h) ft 

20,000 

20,000 

40,000 

10,000 

Mach M 

0.67 

0.40 

1.2 

0.8 

Dynamic pressure 

305 

109 

395 

652 

(q) psf 

Quadratic weights 
qj on Xg 

100 

1,000 

1,000 

100 

qg on Xg = C* - C*c 

10 

100 

100 

1 

qg on 

1,000 

1,000 

10,000 

1,000 

q^ on 

1,000 

1,000 

1,000 

1,000 

Selected control gains 





q to 6 ^ 

0.30 

0.96 

0.19 

0.24 

"z to 6 ^ 

. 0086 

.0011 

.0017 

-.000012 

xg to 6c 

.040 

.13 

.040 

.013 

6c to 6c 

.23 

.11 

.13 

. 056 


TABLE III.- IDENTIFICATION ACCURACY LIMITS FOR THE F-8C 


Parameter 
(from table I) 

True 

value 

Initial 

uncertainty 

1-a 

uncertainty 
at T = 5 

1-0 

uncertainty 
at T = 10 


0 

0.065 

0.063 

0.062 

^2 

0 

.013 

.065 

.056 


0 

31.5 

30.5 

29.7 


0 

5.0 

4.6 

4.4 

M6o 

-13.8 

15.0 

1.46 

1.13 

*^5 

0 

•7^ 

.75 

.75 





TABLE IV.- F-8C IDENTIFIER ACCURACY AT 
FIXED FLIGHT CONDITIONS 


Conditions 

Percent error^ 

Mgo 


As 

V 

q 

a 

h = 20,000, M = 0.67 






Quiet 

<4 

10 

4 

2 

: 5 

Gusts & sensor noise 

7 

<8 

<3 

10 

' 

Pilot commands 

<4 

<8 

6 

2 


h = 20,000, M = 0.4 






Quiet 

<11 

^15 

8 

2 

8 

Gusts & sensor noise 

<11 

<15 

<5 

10 


Pilot commands 

<11 

<15 

8 

4 


h = 40,000, M = 1.2 






Quiet 

<3 

2 

2 

16 

<16 

Gusts & sensor noise 

12 

2 

4 

30 


Pilot commands 

<3 

2 

2 

15 


h = 3,000, M - 0.8 






Quiet 

2 

17 

40 

2 

<9 

Gusts & sensor noise 

23 

9 

23 

15 


Pilot commands 

6 

9 

29 

4 



^Errors were computed by comparing linearized coefficients with estimates 
plotted in real time on strip charts. Many errors fall below the expected 
resolution of this process. 




Figure 1.- Adaptive control law structure. 



Definitions : 

q pitch rate 

a total angle of attack 

elevator position 

6^ elevator connand 

n^ normal acceleration 

unit Intensity white noise 

Tic»v»n„, unit variance white sequences 
gK nz. 


a 

d 



Z5V, V 


Mo* 8/V 


actuator bandwidth (12 r/sec) 
accelerometer dlaplacenent from C.C. 
(15 ft) 

vertical gust level (ft/sec rms) 
sensor noise levels 

dynamic coefficients (functions of _c) 

constant trim disturbances (functions 
of O 


Figure 2.- F-8C longitudinal design model. 
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Figure 3.- Longitudinal optimization problem. 
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Figure 4.- Simplified longitudinal con- 
trol law. 


Figure 6.- Lateral-directional control 
law. 
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Figure 5.- Longitudinal gain. 



Figure 7.- Identification accuracy 
analysis. 


Kalman 

filter 

£■£1 


Accum. ; 
L(£,, t) 


0.2 


s+0.2 






Kalman 

filter 

£•£2 


Accum. 
L(£2, t) 

- 

0.2 

s+0.2 






Kolmon 

fitter 

£*£5 


Accum. 

- 

0.2 

, s+0.2 


L| 


Min 

select 


' 8-33 
update/ 
sec 


ill* 

A 



50 update /sec 8.33 update /sec 


Figure 8.- Parallel-channel tnaximum 
likelihood/identifier . 
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SELF-TUNING REGULATORS 


K. J. Astrom 

Lund Institute of Technology, Sweden 


1. INTRODUCTION 


This paper gives a brief review of the results of a research project on 
self -tuning regulators which has been carried out at the Ltjnd Institute of 
Technology. The project Is part of a larger research program on adaptive 
control which has followed three main lines: A - stochastic control, 

B - self-tuning regtilators, and C - analysis of adaptive regulators proposed 
In literature. 

The approach via stochastic control, which leads to dual-control 
strategies, has been very useful to provide understanding and Insight. So 
far the results have, however, not been carried out to the stage of implemen- 
tation. Self-tuning regulators are a particular version of adaptive 
regulators based on real-time Identification. They are a special case of 
nondual stochastic control algorithms. The work on self-tuning regulators 
has progressed quite far In the sense that these regulators are reasonably 
well understood theoretically. They have also been tried extensively in 
several Industrial applications. On the other hand, much work remains to be 
done In exploring other aspects of these regulators. Project C is needed to 
stay abreast of the development of other adaptive schemes. It has also 
resulted In a long list of problems relating to understanding the strange 
behavior of some algorithms In certain circumstances. 

The basic Idea underlying the self-tuning regulators Is the following. 

If a description of a system and Its environment Is known, there are many 
procedures available to design a control system subject to given specifica- 
tions. When trying to remove the assumption that the models for the system 
and Its environment are known, we are Immediately led to the problem of 
controlling a system with constant but unknown parameters. This problem can. 
In principle, be solved by using stochastic control theory at the price of 
exorbitant calculations. It Is then meaningful to ask If there are simple 
control algorithms that do not require Information about the model parameters, 
such that the controller will converge to the controllers that could be 
designed If the model parameters were known. It Is an empirical fact that 
such controllers exist In several cases. The investigation of their proper- 
ties has also led to powerful tools that can be used to analyze many other 
cases . 


The generation of self -tuning algorithms is partly heuristic. It turns 
out that many algorithms can be obtained by combining a real-time Identifier 
with a control scheme. In our work we have so far mostly considered regula- 
tors for the LQG regulator problem. This has been motivated by the particular 
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applications we have considered. Many of the concepts and Ideas can, however, 
be extended to many other design methods. 


2. AN EXAMPLE 


The main Ideas are first demonstrated by a simple example. Consider the 
simple discrete time system: 

y(t + 1) + ay(t) = bu(t) + e(t + 1) + ce(t) (1) 

where u is the input, y the output, and {e(t)} a sequence of Independent, 
equally distributed, random variables. The number c Is assumed to be less 
than 1. Let the criterion be to minimize the variance of the output, that 
is, 

, t 

min V = min Ey^ = min E — ^ y^ (k) (2) 

k=l 

It is easy to show that the control law 

u(t) = y(t) (3) 

is a Tnin^mnm variance strategy, and that the output of system (1) with 
feedback (3) becomes 


y(t) = e(t) (4) 

(see, e.g., Astrom (ref. 1)). Note that the control law (3), which represents 
a proportional regulator, can be characterized by one parameter only. 

A self-tuning regulator for the system (1) can be described as follows: 

ALGORITHM (Self-TUning REgulator) 

Step 1 (Parameter Estimation) 

At each time t, fit the parameter a in the model 

y(k + 1) + ay(k) = u(k) , k = 1, . . ., t - 1 (5) 

by least squares, that is, such that the criterion 

t 

E €^(k) (6) 

k=l 
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(7) 


where 


e(k) • y(k) - y(k) 

Is minimal. The estimate obtained Is denoted to Indicate that It :|.s a 

function of time. 

Step 2 (Control) 

At each time t, choose the control 

u(t) - a^y(t) (8) 

where Is the estimate obtained In step 1. 


Motivation 

There are several ways to arrive at the control strategy given above. 
The algorithm STURE can, for example, be Interpreted as the certainty 
equivalence control for the corresponding stochastic control problem. 


Analysis 

The properties of a closed-loop system controlled by a self-tuning 
regulator are now discussed. Since the closed-loop system Is nonlinear, time- 
varying, and stochastic, the analysis Is not trivial. 

It Is fairly obvious that the regulator will perform well If It Is 
applied to a system (1) with b = 1 and c = 0, because In this case the 
least-squares estimate at will be an unbiased estimate of a. The regulator 
(8) will thus converge to a minimum variance regulator If the parameter 
estimate converges. It Is surprising, however, that the regulator will 

also converge to the minimum variance regulator If c ^ 0 (as demonstrated 
below). There may also be some difficulties because the control law Is of 
the certainty equivalence type. Because of the special model structure (5), 
the feedback gain will, however, be bounded If the estimate a^ Is bounded. 

The least-squares estimate Is given by the normal equation 

7 52y(k + i)y(k) + a i Ey^(k) = 7 Z)y(k)u(k) 
k-1 '■ k=i ^ k-1 

Asstjmlng that the estimate converges toward a value that gives a stable 

closed-loop system, then it Is straightforward to show that 
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i Z<vi - ’ ° 

Thus the closed- loop system has the property 

t ■ . 

lim 7 2^y(k + l)y(k) = 0 (9) 

t-xo = k»i 

Furthermore, assuming that the system to be controlled is governed by 
equation (1) , the output of the closed-loop system obtained in the limit is 
given by 


y(t) + [a - ab]y(t - 1) = e(t) + ce(t - 1) 
The covariance of {y(t)} at lag 1 is then given by 

Ey(t + l)y(t) - -f(«) - fe - ft 

1 - (a - ab)^ 


Condition (9) gives 


( 10 ) 


( 11 ) 


f(a) » 0 


A second-order equation for a which has the solutions: 


a = Uj 



a - a 


2 


a - (1/c) 
b 


The corresponding poles of the closed-loop system are >= c and 
X 2 = 1/c, respectively. Since c was assumed less than 1, only the value 
corresponds to a stable closed-loop system. Note that corresponds to the 

gain of the minimum variance regulator (3). Hence, if the parameter estimate 
converges to a value that gives a stable closed-loop system, then the 
closed-loop system obtained must be such that equation (9) holds. This means 
that the algorithm can be thought of as a regulator that attempts to bring 
the covariance of the output at lag 1, that is, ty(l), to zero in the same way 
as an integrating regulator brings the integral of the control error to zero. 


If the system to be controlled is actually governed by equation (1) , then 
the self-tuning regulator will converge to a minimum variance regulator if it 
converges at all. 
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Figure 1 shows the results of a simulation of the algorithm. It is clear 
from this simulation that the algorithm converges In the particular case. The 
least-sqxiares estimate will be a biased estimate of the model parameter 
a ■ -0.5 because of the correlation between the model errors. As can be 
expected from the previous analysis, the bias Is, however, such that the limit- 
ing regulator corresponds to the minimum variance regulator. The lower part 
of figure 1 shows the asymptotic value of the loss function obtained If the 
regulator gain Is fixed to the current value. It Is clear from this figure 
that the loss function Is very close to the minimum loss for the case of known 
parameter after 50 steps. 


3. GENERALIZATIONS 


A regulator that generalizes the simple self-tuner of the previous sec- 
tion Is shown In figure 2. The regulator can be thought of as being composed 
of three parts: a parameter estimator (block 1), a controller (block 3), and 

a third part (block 2), which relates the controller parameters to the esti- 
mated parameters. The parameter estimator acts on the process Inputs and 
outputs and produces estimates of certain process parameters. The controller 
Is simply a linear filter characterized by the coefficients of Its transfer 
function. These coefficients are generally a nonlinear function of the 
estimated parameters. This function Is frequently not one to one. This way 
of describing the regulator Is convenient from the point of view of explaining 
how It works. The subdivision is, however, largely arbitrary, and the regula- 
tor can equally well be regarded simply as one nonlinear regulator. The 
functions of blocks 1, 2, and 3 are also simple, but the Interconnection of 
these blocks represents a system with a rather complex Input-output relation. 
The partitioning of the regulator (fig. 2) Is also convenient from the point 
of view of Implementation because the parameter estimator and the controller 
parameter calculation are often conveniently time shared between several loops. 

There are many different ways to estimate the parameters 0 and to 
calculate the regulator parameters, ^ . Some possibilities are shown In 
figure 3. The complexity of the algebraic equation that relates the control- 
ler parameters to the estimated parameters can vary significantly, from a 
simple variable substitution for minimum variance regulators to solution of an 
algebraic Rlccatl equation for the general LQ6 case. 


Analysis 

A brief statement of some properties of the self-tuning regulators are 
now given. The results are fairly techhlcal and only a few main points are 
given here. A review of available results are given In reference 2. The 
major results were proven In references 3 to 5. 

For the analysis. It Is assumed that the process to be controlled Is 
governed by 
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( 12 ) 


A(q"by(t) » B(q“^)u(t - k) + v(t) 

where A(q and B(q are polynomials in the backward shift operator q"”^ 
and {v(t)} is a sequence of random variables with bounded fourth moment. 

The analysis will basically cover the case y(t) -»• 0 as t which corre- 

sponds to the case when the parameters are constant. 

The following problems can be resolved partially by analysis: 

• Overall stability of the closed-loop system 

• Convergence of the regulator 

• Properties of the possible limiting regulators 

The analysis Is far from trivial because the closed-loop system Is a nonlin- 
ear, time-variable stochastic system. Even If the recursive Identification 
schemes used are well known, their convergence properties are largely unknown 
except for the least-sqxiares case. The Input Is also generated by a time- 
varying feedback, which Introduces additional dlfflcult|.es. If the process 
noise {v(t)} Is correlated, the least-squares estimates will be biased and 
the bias will depend on the feedback used. 

A global stability result was proven by LJung and Wlttenmark (refs. 5 
and 6) (see fig. 4). This result applies to a regulator composed of a least- 
squares Identifier and a mlnlmtan variance controller. The result requires 
that the system (12) Is minimum phase and that the time delay k and the 
parameter 3^ are known. 

A key result In the analysis Is the observation made by Ljung (ref. 4) 
that the paths of the estimates are closely related to the trajectories of 
the differential equation: 


where 


f = Sf(0) 

^ = G(0) - S-1 


(13) 


f(0) = E[’l'^(t,0)e(t,0)] 
G(0) = E[H'^(t,0)’l'(t,0)] 


(14) 


In the particular case of the regulator LS4MV, the control law Is chosen 
In such a way that y(t,0) = 0 and the stationary points are then given by 

0 = f(0) = E[y(t + l)<{i(t)] = 0 
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The regulator LS4MV thus attempts to zero the auto covariance of the output 
and the crosscovariance of the input and the output for certain lags. This 
result, which generalizes the simple example discussed in section 2, was shown 
in reference 7. It was also shown here that 

f(0) = 0 (15) 

has only one stationary solution for the regulator of LS+MV if the orders 
of the system and the model are compatible. 

The differential equations (13) and (14) can be used in several different 
ways. Ljung has exploited them to construct both convergence proofs and 
examples which show that the parameter estimates do not converge. The dif- 
ferential equations have also been very useful in simulations (see, e.g., 
refs. 8 and 9). 

The simulations shown in figures 5, 6, and 7 illustrate the behavior of 
different versions of the self-tuner. 


4 . SERVOPROBLEM 


So far, the self-tuning regulator has been discussed only in the frame- 
work of the regulator problem. It is straightforward to apply self-tuning 
to the servoproblem, too. Clarke and Gawthrop (ref. 10) propose to do so by 
posing a linear quadratic problem for a servoproblem. 

Another approach is simply to introduce the reference values by the 
standard procedure using feedforward and an inverse model. For known param- 
eters, the problem is handled as follows. Assume that the process is 
described by equation (12) and introduce the reference values u^^(t) and y^(t) 
which satisfy the same dynamics as the process 


A(q~^)y^(t) = B(q“^)u^(t - k) 


(16) 


Hence 

A(q"^[y(t) - y’^Ct)] = B(q"^[u(t - k) - u'^Ct - k) ] + v(t) 
A design procedure for the regulator then gives the feedback 


u(t) - U^'(t) = [y(t) - y’^(t)] 

F(q"^) 

If the command signal y (t) is specified, 

u(t) « yf(t + k) + [y(t) - y’^(t)] 

B(q"^) F(q"l) 


( 17 ) 
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This system cannot be realized unless the change in reference value is known 
or can be predicted k steps ahead. If this is not the case, a time delay 
in the response of k units must be accepted. 

Observe that the control law (17) can be written: 

B(q~bF(q"bu(t) = B(q"bG(q"^)y (t) + [A(q"bF(q“^) - q^'^GCq"^ Jy’^Ct + k) (18) 


The servoproblem can be incorporated into the self-tuning regulator simply 
by changing the model in the parameter estimation step to 

Mi y(t) = - i4(q"^)y(t - 1) + 5(q“^)u(t - k) + d7(q”‘^)e(t - 1) 

+ ^5(q ^)y^(t - 1) 

and making the modification (18) in the control step. 


5. APPLICATIONS 


The self-tuning regulators are conveniently implemented using a digital 
computer. The simple regulator LS+MV requires no more than 30 lines of 
FORTRAN code, while the regulator RML+LQ requires an order-of-magnitude more 
code because of the necessity of solving the algebraic Rlccatl equation in 
each iteration. The regulators have been applied to a number of Industrial 
processes. Among the applications currently known to me are 

• paper machine (refs. 11 and 12) 

• digester (ref. 13) 

• ore crusher (ref. 14) 

• enthalpy exchanger (ref. 15) 

• supertanker (ref. 16) 

Several of these applications have been in operation for a long time. A self- 
tuning regulator has, for example, been running as an adaptive autopilot for 
a supertanker for more than a year. 

Even if the regulators discussed automatically tune their parameters, 
it is necessary to determine some parameters in advance; for Instance, 

• Number of parameters in the prediction model (p, r, and s) 

• Initial values of the parameter estimates 

• Value of any fixed parameters in the model 

• Rate of exponential forgetting of past data in the estimation algo- 
rithm 

• Sampling rate 

Experience has shown that it is fairly easy to make the proper choice 
in practice. These parameters are also much easier to choose than to directly 
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determine the coefficients of a complex control law. It is our experience 
that system engineers without previous exposure to this type of algorithm have 
been able to learn how to use it after only a short training period. There 
have also been several misapplications. The most common mistake is to attempt 
a self-tuner for a control design that will not work even if the parameters 
are known. 
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Figure 1.- Example of self-tuning regulator. 



Jj^egulotor 


Figure 2.- Block diagram of a general self-tuning regulator. 
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PARAMETER ESTIMATIOM 


Model: y(t) - -A(q ^)y(t - 1) + B(q ^)u(t - 1) + C(q ^)e(t - 1) 

- <>(t)0 

e(t,0) “ y(t) - i^(t)0 
0(t + 1) - 0(t) + ji(t)S(t + l)i|;’^(t)e(t,0) 

S“^(t + 1) - S"\t) + u(t + l)[/(t + + 1) - S"^(t)l 

For: Least squares, C = 0 

Extended least squares, 4>(t) * ifiCt) 

Recursive maximum likelihood, -grad0e(t,0) 

CONTROL STRATEGIES 

y(t) - y(t) 

F(q"') 

Regulator parameters: ^ * col[q^q 2 . . . q^, ; f^, f 2 ••• ^ 2 ^ 

Criteria: Minimum variance 

o 

Linear quadratic 


Figure 3.- Some approaches to parameter estimation and control. 
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Let the system be 


^(q"by(t) = S(q"^)u(t - k) + V(t) 

where the parameters are estimated by least squares 
and control gives a minimum variance response 
If the time delay, k, and the lead coefficient of 

the poljmomlal S(q ), are known 
If the system order Is not underestimated and 
If 


11m SUP ^Sv^(t) < ~ 

then . 

11m SUP ^2y (t) < «> 
N 

and if the system be minimum phase then also 

llm SUP ~ 

N 


Figure 4.- An example global stability result. 
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FAILURE DETECTION AND CONTROL- SYSTEM RECONFIGURATION: 
PAST, PRESENT, AND FUTURE 
Raymond C. Montgomery 
Langley Research Center, NASA 


SUMMARY 


The history of failure detection and redundancy management in aircraft 
applications is reviewed. To date, techniques related to that subject have 
been based mainly on hardware duplication of like components with failure 
monitoring and switchover or averaging for redundancy management. Specific 
examples of these techniques are discussed as they have been, applied to the 
NASA F8 Digital Fly-by-Wire (DFBW) aircraft and are to be applied to the space 
shuttle vehicle in the near future. 

Recently, interest has arisen in new mathematical developments that 
promise optimal failure detection and control law reconfiguration through the 
use of system dynamical equations and measurements from dissimilar components. 
Practical use of those developments depends on the availability of advanced 
logic processing such as is offered by modern high-speed computers. Recent 
results of simulations using those developments are presented. They indicate 
that, in the simulation environment, failure detection and control-system 
reconfiguration through the use of dissimilar components is possible and func- 
tions satisfactorily for sensors used for primary flight control. However, 
computational requirements for the system studied are large and exceed the 
capability of the F8-DFBW aircraft as presently configured. Due to the 
parallel nature of the algorithm needed for the system, the use of micropro- 
cessors as dedicated computational units should alleviate this problem. For 
the time being, however, to develop advanced redundancy management concepts to 
a technology application level, NASA is proceeding with the development of 
advanced suboptimal systems. Plans for a flight test of the advanced redun- 
dancy management system are presented and discussed. 


PAST 


Since the advent of aviation there has been concern regarding safety of 
flight. That concern is most apparent in the aircraft flight-control system. 
The purpose of most flight- control systems is to translate motions from the 
pilot *s controllers into appropriate motions of control surfaces located on 
the aircraft. In the early days of aviation, this was accomplished by an 
all-mechanical control system as indicated in figure 1(a) . That type of sys- 
tem is in use today in most light aircraft. One feature of the mechanical 
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system is that loads required to move the control surfaces are transmitted 
through mechanical linkages to the pilot, thereby giving him a feel of the air- 
craft. As aircraft became larger, the forces transmitted also became larger. 
That led to the requirement for the fully powered hydraulic system shown in 
figure 1(b). The forces applied to the control surfaces are not felt by the 
pilot in the fully powered system so that artificial feel and trim had to be 
used. Also, because of dependence on the actuation system, increased reliabil- 
ity was needed. This led to tandem actuators, multiple hydraulic supply sys- 
tems, hydraulic voting mechanisms (usually force sum devices) and many other 
ad hoc procedures to meet the vehicle design requirements from the reliability 
standpoint. 

o 

Desire to have aircraft fly in conditions where its flying qualities are 
undesirable led to the development of the stability augmentation system (SAS) 
shown in figure 1(c). That system uses sensors to detect aircraft motions and 
electrical signals determined from the sensors to activate the hydraulic 
system. The signal is applied in addition to the control commands from the 
pilot’s controls. The SAS system introduced new requirements for reliability 
since failure of the sensors could adversely affect the vehicle’s motions. 
Again, these were resolved largely through ad hoc procedures usually involving 
limiting the authority of the SAS inputs. This has been done by a variety of 
methods usually employing ingenious mechanical devices (or nightmares, depend- 
ing on your point of view). One approach to eliminate the complex, weighty, 
and maintainably undesirable mechanized nightmare is being developed by 
government and industry— the fly-by-wire control system (shown in fig. 1(d)). 
That system will employ electrical wires to replace the complex mechanical 
linkages indicated above. It enables a flexible design with advantages of 
weight saving and simplicity. The main reasons that such systems are not 
used in operational aircraft today is that there is a general lack of confi- 
dence in the basic reliability of the electrical and electronic components 
used in a fly-by-wire design. An increase in the basic reliability of the 
fly-by— wire system will be necessary before it can be used operationally. 

NASA is involved in two programs to develop fly-by-wire technology. At 
the Dryden Flight Center (DFRC) , a digital fly-by-wire system is being 
installed onboard an F8 aircraft as part of the F8-DFBW project. Indeed they 
are involved in the second phase of their activity, having completed a phase 
where the APOLLO digital computer was used as the central control unit in a 
fly-by-wire system. The current phase of the project involves replacing the 
APOLLO computer with a set of IBM AP-101 computers. Concerning reliability 
requirements for the basic fly-by-wire mechanization, it was determined that 
fly by wire required quadruply redundant secondary actuators. The actuator 
concept used in the first phase of their work is illustrated in figure 2 
(ref. 1). The actuator is constructed with three tandem pistons’ on a common 
shaft. It is powered by two independent hydraulic supplies. It has four 
independent servosys terns to control the position of the actuator. Normal 
operation uses servosys tern 1, which consists of an active and a monitor, 
.two-stage, flapper nozzle servo valve. The active valve is monitored for 
failure by the monitor valve using a hydraulic comparator. If the difference 
in output of the monitor and active unit exceeds a threshold, the supply pres- 
sure engaging the active valve will be dumped to return (see slide valve at 
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lower part of servosystem 1 block). This will occur with motion of the com- 
parator spool in either direction. Simultaneously, the hydraulic engage valve 
for servosystem 3 is hydraulically enabled. The primary channel fail switch 
causes all remaining (servosystems 2 and 3) engage solenoids to be energized. 
Thus the actuator is configured as a force summed triple-tandem actuator when 
all engage valves are engaged. Additional failures will result in some 
degradation of performance. 

The real question is: what is the cost of providing redundancy in the 

form of hardware duplication? Some measure of that is obtained from the 
actuator example. This resulted from the existence of ground simulators 
required for F8 project support. Because of the lack of a requirement for 
quadruplex actuators in the simulator environment, Langley Research Center 
procured simplex actuators. The cost of the quadruplex actuators is approxi- 
mately $20,000 per actuator, whereas simplex actuators are available at one- 
quarter the cost. In addition to increased cost, there also remains a ques- 
tion of performance degradation in redundant systems. When Langley acquired 
their actuatots, it was determined that their bandpass had to be reduced from 
50 to 10 Hz to simulate the quadruplex actuators onboard the aircraft. 


PRESENT 


The other NASA activity to develop fly-by-wire technology is the space 
shuttle project. Early in the project, it was determined that mission 
requirements for the shuttle dictated fly by wire. In addition to flight con- 
trol requirements, the space shuttle has navigational reliability requirements 
that dictate advanced failure detection and redundancy management techniques 
not previously used. To meet reliability requirements, the shuttle avionics 
system evolved to a hardware configuration that uses hardware duplication for 
the different systems. For example, in hardware, the rate gyro assembly is 
duplicated three times and the computers five times. Management of the hard- 
ware redundancy for the components, a major problem, will be done by the 
computers. The computers operational flight program is divided into modules 
with all hardware interface handled in a module referred to as subsystem 
operations. That module acts as a system executive and controls all input/ 
output interfaces. It also passes data from the I/O interface to the appli- 
cations programs (guidance, navigation, flight control, etc.). A redundancy 
management module determines which data are to be passed to the remaining 
applications programs and to hardware. The techniques for doing that depend 
on the particular type of data being processed. Techniques used for sensor 
data, controller data, switch data, and actuator data tracking tests, builtin 
stimulus tests (BIST), and comparison tests. One unique feature related to 
redundancy management, not used in past aircraft applications, is the handling 
of the navigational data (fig. 3). Three inertial measurement units are used 
in parallel to generate position and velocity estimates. The position and 
velocity estimates are then adjusted for new navigation data input by the 
navigational sensors using a Kalman- Bucy-type filter. A data selection 
algorithm is then used to determine data to be passed on to the remaining 
applications programs, namely, the guidance and control modules. 


71 


Methods used by the shuttle system to select data depend on the level of 
data redundancy available. Candidate three-level methods are being considered 
which include simple average, sample middle select, Kaufman *s weighted aver- 
age, and Landley*s method. Two-level methods being considered include aver- 
aging and voting using an alternate data source (i.e., using, say, NAV module 
generated pitch rate to determine which of two pitch rate gyros is functioning 
properly). In the two-level case, there was a possibility of including an 
advanced redundancy management algorithm based on early elements of decision 
theory. For the two-level case, a filter using Waldos sequential probability 
ratio test is being considered. Although it offers the potential of improved 
redundancy management (especially for small bias errors on IMU data), the 
method was computationally burdensome. It also relied heavily on the prior 
statistics determined from sensor bench tests. 

In addition to the navigation module, there is, of course, a requirement 
for management of redundancy in the more classical fashion for the shuttle 
for the controls and sensors. One proposed method being evaluated for han- 
dling the sensor redundancy is: under no failures, middle value select will 

be used with tracking test to detect failures. Failure thresholds will be 
taken about the selected signal. A failure to track within the threshold is 
considered to be a failure of the device if it fails to track for N consecu- 
tive samples nominally spaced at 40-msec Intervals. After the first failure, 
the failure is displayed to the crew and tracking tests are mechanized the 
same as in the no failure case, except that thresholds are taken about the 
average of the two good sensors. Again, failure to track assumes that a 
failure has occurred if it occurs for N consecutive samples. The action to 
be taken following a second failure is to notify the crew and safety select 
a single unit. BIST would then be initiated on the remaining unit. If it is 
determined to be a good unit, an Interchange would then be made of the 
selected unit. 


FUTURE 


As is being applied in the shuttle project, one important potential for 
digital control systems is their ability to reorganize themselves following 
failures in sensors and actuators. For sensors, this has been accomplished 
using duplication of components and a simple voting process to manage the 
redundancy. Redundancy provided in that way can be termed "hardware” redun- 
dancy since it owes its existence to hardware duplication. Redundancy also 
exists between dissimilar components. However, to use that type of redun- 
dancy, one must be aware of the dynamic behavior of the system and the inter- 
actions between system components. Such redundancy is more properly termed 
"analytical" redundancy since it owes its existence to analytical knowledge of 
the systems behavior. Langley Research Center is developing this type of 
redundancy management for use in aircraft flight control. It is anticipated 
that a system will be developed and flight tested on the F8-DFBW aircraft at 
FRC. Simulation tests of an "analytical" redundancy management system for 
that aircraft have been made at Langley. The theoretical approach used and 


72 



portions of the results of those tests are described here. The approach is 
reported in detail in reference 2 . 

To use analytical redundancy requires that the designer have an analyt- 
ical model that expresses the response of the system to control command inputs. 
It also requires consideration of the level of confidence that the designer 
has in the modeling process. Figure 4 shows the form of the model used for 
the F 8 work. Basically, the model is a complete nonlinear six-degree-of- 
freedom modiel of the aircraft's motion which has a good kinematic representa- 
tion using quaternions but uses an aerodynamic force representation that 
employs linear aerodynamic coefficient modeling. That is, the force and 
moment coefficients are linear functions of angle of attack and control posi- 
tions, Also note that the F 8 -DFBW is a digitally controlled aircraft and 
hence requires a discrete form representation of the aircraft dynamics. That 
discrete form was obtained using Euler's method using a step size of 1/32 sec 
to be consistent with Langley's real-time computer system. The selection of 
the statistics was made considering that kinetic relations are more likely to 
be accurate than aerodynamic force and moment relations. 

Figure 5 outlines the approach taken for failure detection. That 
approach is derived from Bayesian decision theory and involves selecting a 
group of hypotheses that are checked for validity. The different hypotheses 
represent different assumptions regarding the status of different subsystems. 
For example, hypothesis Hq might represent the assumption that all systems 
are operating normally. Hypothesis might represent the hypothesis that a 

specific sensor has failed. Hypothesis H 2 might represent the hypothesis 
that a multiple failure has occurred where, say, both the roll and yaw rate 
gyros have failed. The rationale for making a decision is to construct a 
function that assigns a cost to making an incorrect decision and then to mini- 
mize that function. The weights shown on the slide are the weights that 

the designer assigns to deciding on hypothesis Hj as true. Of course, if 

i = j , the decision is correct so that normally tne terms of the form 

are taken to be zero. Figures 6 and 7 show simulation results that indicate 
the importance of the selection of the weighting coefficients. The results 
are generated from a simulation of an entry vehicle. Figure 6 shows the 
response of the vehicle with no augmentation on the left; on the right is the 
input response with augmentation. It is that response that we will look for 
in figure 7 , which shows one continuous run of the simulation where different 
hypotheses are assumed and where a controller designed for the assumed hypo- 
thesis is used. In that run, Hq is the unfailed hypothesis, H^ is failure 
of the roll rate gyro, H 2 is failure of the yaw rate gyro, and H 3 is 
failure of an angle-of-sideslip Indicator. During the entire run, the yaw 
rate gyro was failed by increasing the noise superimposed on its output. Note 

the character of the response when that hypothesis (i.e., H 2 ) is used. 

The resolution to the Bayesian hypothesis testing problem involves two 
items (fig. 8 ). First one must construct hypothesis-conditioned estimates 
of the aircraft's state for each hypothesis selected. This implies having a 
bank of conditional mean estimators parallel in operation. The second step 
involves examination of the performance of each hypothesis-conditioned 
estimator considering the performance realizable if that hypothesis is used. 
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This is mechanized as a test of the residuals of each estimator with weights 
that depend on the expected error covariance of the estimates. 

/ 

A block diagram of a failure detection system that might be implemented 
on an aircraft is indicated in figure 9. To implement the procedure Requires 
computational capability in excess of that available on flight computers of 
today. Figure 10 lists some practical considerations used to study /the con- 
cept. Simulation results to be presented were generated on the F8-Simulatlon 
facility at Langley which involves a complete piloted six-degree-of-freedom 
simulation. The aerodynamic representation used in the simulation accounts 
for effects of Mach number, altitude, and aeroelasticity. Concerning the 
failure detection system, it used a nonlinear prediction model obtained using 
linear aerodynamics (as mentioned earlier) . Filter gains for the different 
hypotheses-conditioned estimates were obtained using steady-state Kalman 
filter gains obtained at level flight equilibrium at 20,000-ft altitude and 
a Mach number of 0.6. Figure 11 shows the performance of that system when 
the simulated F8 aircraft is subject to a rapid roll maneuver. In these runs, 
the hypothesis tested were Hq “ no failures, — roll rate gyro failure, 
and H 2 — yaw rate gyro failure. Traces of the actual and estimated failure 
state are shown on the slide. 

NASA currently has a contract with the Draper Laboratory to mechanize 
and study requirements for an analytical redundancy management system relative 
to an F8-DFBW mechanization. It is anticipated that the output of that con- 
tract will lead to a flight test. The concept to be mechanized will be con- 
strained by computer requirements and will hence probably not be a mechaniza- 
tion of the bank of filters approach being studied at Langley. 

Systems employing banks of filters are items for the future. They will 
probably most easily be mechanized using dedicated microprocessors. Langley 
is initializing work on optimal decentralization of flight-control processes 
that use microprocessors as dedicated control units and signal processors. 

One experiment being considered is the construction of a microprocessor con- 
trol unit for a digital actuator test facility at Langley. That control unit 
would provide for actuator stabilization, failure detection, and redundancy 
management. It would, in conjunction with other processors, provide an 
advanced redundancy management system using ideas presented here. 
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HYPOTHESIS SEIICTION 
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Figure 5 .- Analytical redundancy man- 
agement using multiple hypothesis 
testing. 
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Figure 6.- Roll responses for simulated 
re-entry vehicle. 
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Figure 8.- Minimum Bayesian risk solution for 
analytical redundancy management. 






Figure 9.~ Analytical redundancy management system. 
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the F8 DFBW aircraft. 
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FACTS AND FICTION OF LEARNING SYSTEMS* 
George N. Saridis 
Purdue University, Indiana 


1. INTRODUCTION 


The latest developments in computer technology and the ensuing successes 
of space exploration has generated a widespread belief that the aircraft and 
spacecraft of the future will be completely automated machines that may 
reliably accomplish preprogrammed treks and explorations in our galactic 
system. 

It has been also widely assumed that modem optimal control theories pro- 
vide the appropriate methodology to generate such automated controls. This 
may be so, but only in the far future when the computer technology would have 
progressed sufficiently more in sophistication and miniaturization. However, 
to meet the realistic demands of modern highly sophisticated high-speed air- 
craft and spacecraft requiring for their operation advanced system reliabil- 
ity, a man-machine interactive control system represents the optimal design. 
Such a system would require high-precision electromechanical controls as well 
as higher level decision-making and task coordination and planning, accepting 
only qualitative commands and task override by the human operator. Reliabil- 
ity of this nonconventional control system requires almost anthropomorphic 
functions for the controller such as training capabilities for the hardware 
for performance improvement in the presence of uncertainties, software optimal 
decision-making, and interface with the human operator. 

The methodology that will provide the updated precision for the hardware 
control and the advanced decision-making and planning in the software control 
is called "learning systems and intelligent control." It has been developed 
theoretically as an alternative for the nonsystematic heuristic approaches of 
artificial intelligence experiments and the inflexible formulation of modern 
optimal control methods. Its basic concepts are discussed in the sequel and 
some feasibility studies of some practical applications are presented. 


2. LEARNING SYSTEMS, SELF- ORGAN I ZING, AND INTELLIGENT CONTROL 


The concept of "learning" has been used in system design and control 
whenever the a priori information of the underlying process is ui^certain or 
unknown (ref. 1). This concept may be applied with equal effectiveness to 
identify uncertain parameters or improve directly the performance of the 


*This research was supported by NSF Grant GK-36607. 
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system by appropriate higher level decision-making which is sequentially 
updated by previous experience. 

The following definitions regarding learning systems and learning control 
systems have been formulated by a subcommittee of Control Systems Society of 
IEEE and are presented here for completeness (ref. 2) . 

Def'Lntt'Lon 1. A system is called teaming if the informa- 
tion pertaining to the unknown features of a process or its 
environment is acquired by the system, and the obtained 
experience is used for future estimation recognition, classi- 
fication, decision or control such that the performance of 
the system will be improved. 

Definition 2. A learning system is called a teaming oontvot 
system if the acquired information is used to control a pro- 
cess with unknown features. 

Note that learning systems represent a wide class of processes, wider 
than control systems. As such they may serve as high-level decision-makers, 
problem solvers, and task planners and organizers in artificially intelligent 
systems like autonomous robots, interactive man-machine systems, bionic 
devices, and pattern recognizers. Such learning systems that may perform 
on-tine as well as off-tine are characterized as intettigent systems or con- 
trot and are intended for highly sophisticated anthropomorphic functions. 

When learning is applied to control hardware systems, one may devise a 
controller that would on-tine improve its performance regardless of uncertain- 
ties governing its mathematical modeling. Such control systems have been 
characterized as set f -organizing oontrot systems and are defined in refer- 
ence 2 as follows: 

Definition 3. A control process is called setf-organizing 
if reduction of the a priori uncertainties pertaining to the 
effective control of the process is accomplished through 
information accrued from subsequent observations of the 
accessible inputs and outputs as the control process evolves. 

A controller structured to perform the above task on-tine is 
called a set f- organizing controtter. 

Definition 4. A self-organizing control process is called 
parameter-adaptive if it is possible to reduce the a priori 
uncertainties of a parameter vector characterizing com- 
pletely the process through subsequent observations of the 
accessible inputs and outputs of the system as the control 
process evolves. A controller structured to perform the 
above task on-tine is called a parameter-adaptive S.O. 
controtter. 

Definition 5. A self-organizing control process is called 
performance- adaptive if it is possible to reduce directly the 
uncertainties pertaining to the improvement of the performance 
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of the process through subsequent observations of the acces- 
sible inputs and outputs of the system as the control pro- 
cess. A controller structured to perform the above task 
on-line is called a performance- adaptive S.O. controller. 

It is obvious from the previous discussion that learning systems cover a 
wide area of decision-makers, ranging from highly sophisticated intelligent 
machines to advanced control systems. Generally, however, they deal with com- 
plex processes for which simple classical control techniques cannot be 
applied. Actually, this is one of the reasons why learning systems have not 
been popular with scientists and engineers until recently. The processes 
under their consideration have been too simple and too well defined to require 
any advanced decision-making. 

In the last few years, the study of large-scale systems in "soft 
sciences," energy environment, space technologies, etc., have found the exist- 
ing analytic tools inadequate. Therefore, scientists started looking for new 
methodologies, one of them being "learning systems," which provide a new 
dimension in the various control problems. In particular, in aircraft and 
space technology with the new high-speed sophisticated aircraft and space 
vehicles operating in unknown environment, two main reasons prompted the use 
of self-organizing control as a learning system for reliable operation: the 

uncertainties involved with the mathematical model and the need of lower-order 
models to represent high-dimensional systems . 

Several feasibility studies have proven the advantages of such control- 
lers and certain actual tests are already in progress by Honeywell and will be 
reported in these proceedings. 

A review of the most successful self-organizing control methods is pre- 
sented in the next section, while two feasibility studies performed by the 
author and his colleagues are discussed in the sequel. However, a word of 
caution is appropriate here; learning systems and self-organizing controls 
should not be used to drive systems for which a simple controller designed by 
classical methods is adequate. Sophisticated techniques are needed only when 
complex systems require them. 


3. A REVIEW OF SOME SELF-ORGANIZING CONTROL METHODOLOGIES 


Numerous methods that qualify under self-organizing control definitions 
have appeared in the literature since 1950. Most of these methods have 
appeared in expository publications (refs. 3-17) under various names such as 
adaptive, optimalizing, self-optimizing, etc., systems. They may be sub- 
divided into two categories, the first one treating systems operating in a 
deterministic environment and the other for systems operating in a stochastic 
environment. The first category covers systems like Margolis and Leondes’ 
parameter tracking servo, the polynomial expansion methods, the automatic 
spectrum analyzer approach, the functional analysis approach, the model refer- 
ence adaptive control methods, error correcting methods, and many others 
(ref. 12). 


81 


The second category, which is the most realistic of the two since it 
incorporates in the formulation the irreducible uncertainties introduced by 
the environment, covers a collection of algorithms with various degrees of 
uncertainties about the plant dynamics. In the subcategory of parameter- 
adaptive self-organizing (SO) controls (ref. 12), one may have to select among 
the linearized stochastic optimal-control algorithms (ref. 18) , the open-loop 
feedback optimal control algorithms (refs. 19 and 20), the Bayesian learning 
algorithms (refs. 21-23), the parallel identification and control (refs. 24 
and 25), and the actively adaptive control algorithm (ref. 26). The, charac- 
teristic of: this subclass is that a parameter identification is necessary to 
provide the information about the uncertainties of the process. In the sub- 
class of performance adaptive. SO controls, the choice may be made among the 
crosscorrelations approach (refs. 27 and 28), stochastic automata methods, 
(refs. 29 and 30) , .the stochastic approximation algorithm (refs. 31 and 32), 
the expanding subinterval algorithm (ref. 33), etc. The characteristic prop- 
erty of this subcategory is that the search for the reduction of uncertainties 
pertains to the evaluation of the performance criterion which indicates the 
improvement of the performance of the systems. Review monographs that give a 
detailed account of the formulation of these algorithms have recently appeared 
in the control literature (refs. 7 and 12), and several applications in man- 
machine and robotic systems as well as spacecraft control (refs. 19 and 34) 
have been discussed. Since the space is rather limited here, no detailed 
account is given and the interested reader is referred to the literature. 
Instead, two space vehicle studies, utilizing SO control techniques and being 
produced by the author and his colleagues as feasibility studies, are pre- 
sented in the sequel in terms of simulation results. 


4. PARi^TER-ADAPTIVE SO CONTROL FOR BOOSTER STAGE OF A SPACE VEHICLE 


Parallel identification estimation and control is used as a typical 
parameter-adaptive SO control scheme to demonstrate the feasibility and appli- 
cation of the method to a booster . stage of a Saturn space vehicle (ref. 24). 

In this application, the goal is the simultaneous flight parameter identifica- 
tion and control of the vehicle during its mission, that is, from launch to 
stage separation while operating in a stochastic environment, generated by 
external wind gusts and control input disturbances. 

Considering a pitch plane motion only, the booster is shown diagrammati- 
cally in figure 1, were a is angle of attack (deg), 6 is attitude angle 
(deg) , and 6 is gimbal angle of the rocket engine thrust chambers (deg) . 

All of the above variables are perturbation angles from a reference trajec- 
tory. Thus the control problem will be one of maintaining small deviations 
about a nominal flight path. For small-angle variations and other simplifying 
assumptions, the rigid-body equation of motion and the d 3 mamics of the engine 
actuator are given, respectively, by 

d^0 Q X /i\ 

- yoi0 = (1) 
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II + kc6 “ 6c^C 


( 2 ) 


where and are unknown time-varying flight parameters and 6 q is the 
gimbal command (in deg) and l/k^ is the engine thrust chamber gimbaling time 
constant. As it stands, the rigid-body equation of motion in equation (1) 
represents an oaerodynamically unstable vehicle. Therefore, a stabilization 
autopilot is initially synthesized using ”worst-case” flight parameters 
and P 5 to ensure stability of the uncontrolled system throughout its flight 
profile. Generally, the values for the worst-case flight parameters are known 
from prior launch data. Figure 2 is a block diagram of the engine actuator- 
vehicle dynamics and autopilot gains. The constants k^, kj^, and Ilq are 
chosen to ensure stability of the overall system for worst-case flight param- 
eters. The variable 9 q represents the attitude command input and is used 
to control the state of the system. 


Letting q = 0, the system in figure 2 can be written in state variable 
form as 


• 

0 
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o 

iH 
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q 

= 

Ma 0 ^5 


q + 0 

. 

5 







+ y + w) 


(3) 


where 6 q represents the feedback control input. The newly introduced terms 
y and w correspond to the perturbation noise input and system disturbance 
input, respectively. Using attitude and rate gyros, the output measurement 
equation for the system in equation (3) is given by 



(4) 


The two output measurement signals are therefore contaminated by additive 
noise disturbances to account for external wind gusts. 


Using the numerical values in reference 24 and a sampling time of 
0.005 sec, the system is converted to the discrete-time form: 


e(k+i) 


1.0009 

0.0049794 

0.002213 


e(k) 

q(k+ 1) 

= 

0.29102 

0.98778 

0.8546 


q(k) 

6(k+l) 


-0.48792 

-0.028015 

1 

0.8052 


6(k) 


+ 


0.0003998 

0.23582 

0.48036 


[0^(k) + y(k) + w(k)] 


(5) 
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( 6 ) 


z(k) 



■0(k)’ 

q(k) 

.fi(k). 


+ V(k) 


where 


z(k) = 

Zj(k) 

and v(k) = 

ViCk)" 

• 

Z 2 (k) 


V2(k) 


The random variables y(k), w(k) , and v(k) are assumed to be jointly 
independent and have Gaussian distributions with the statistical properties: 


E[y(k)] = E[w(k)] = E[v(k)] = 0 
E[y(k)y(j)] = 


E[w(k)w(j)] = 0.016j^^ , E[v(k)v (j)] = 


0 . 016 j^j 
0 


0 

0 . 016 ]jj 

kjj ~ Oj 1) 2, ••• 

For this problem, the performance criterion to be minimized is given by 

3000 

i=0 ^ 


r ( 7 ) 


( 8 ) 


where 


2 0 0l 


Q = 


0 


2 


0 


.0 0 2j 


r = 10 


Saturation bounds on 0^; were set at ±5° to ensure that the system variables 
would not become unrealistically large during the period of Identification, 


To set the matrices in a form suitable for Identification, the following 
transformation is necessary for which the controllability and observability 
subindices are 


Pi 

Using the selection procedure 


— 1» P 2 “2, — 3 

in equation (6) , define a matrix 


(9) 
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■»T' 


1 

0 

. 0 

p = 

hi 

= 

0 

1 

0 


h^A 


0.291 

0.9877 

0.8546 


- 


_ 


- 


Through the following change of coordinates 



x^ (k) 


’e(k) 

x(k) = 

*2 (k) 

= P 

q(k) 


Xg(k) 


-S(k) 


the system (eqs. (3) and (4)) Is transformed into the canonical form: 


( 10 ) 




Pi 

P2 


1 


1 

\ ' 0.002419 

0.0026 


x(k+l) = AgX(k) + BQ[0Q(k) +y(k) +w(k)] = 

0 

1 0 

1 

x(k) 


-0.3591 

1 -0.8183 

1.7946 



+ 


'0.00039986' 

0.23582 

.0.643 


[0^(k) + y(k) + w(k)] ; 


m = 2 


z(k) 



= H^xCk) + v(k) 


(12) 


The parallel identification, estimation, and control scheme is presented 
in figure 3. Observing that system (10) is completely controllable and com- 
pletely observable and that if the random variables y(k), w(k) , and v(k) 
have finite moments up to the fourth order, the identifier will produce esti- 
mates on the matrices Aq and Bq of system (1) by use of a stochastic approx- 
imation algorithm such as 


(k+g) = 6i(k- 1)+Y 
_ ^T(k+g-l) 
k = 1, g + 2, 


/k- l\ (k+g-1) 

^g+l^^(k) 

6i(k- 1)] , 

2g + 3, . . . ; 


[2i(k+g) - - 1) 

i = 1, 2, . . . , m 


\ 


> (13) 
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where 


[ T T T *1 

h° h° h° Af ; 

[ ’ T T T “I 

h° Dq, h° A^Dq, .... h° Af-^Do ; 


^ 3-) 2 f •••} TA 


Do = B. 


Yjk)(k+8-l) = [y (k+g-1), y^(k+g-y2), ..., y^(k)] 

u'Jk)(k + g-l) = [u'^^(k+g- 1), u’^(k+g-2), ..., u’’^(k)] 

W^k) 8 - 1) = [w^(k+g-D, w^(k+g-2), ..., w^(k)] 
The term in equation (10) has been partitioned as 


Ho = [kj. ^2 h^] 

and 0j^(O) is chosen arbitrarily. For convergence of equation (11), 


lira r(i) * 2 2 Y^(i) < 

i->oo i=2 i=l 


E[ll 0^(0) 1|2] <0, i = 1, 


m 


(14) 


The original parameters are recovered by the following general transformations 
used here for m = 2: 
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(16) 


A quick glance shows that the above algorithms are dependent on knowledge 
of the statistics of the noises w(k) , v(k) , y(k), or of the matrix Dq. The 
structure of the feedback controller can also attain an arbitrary form; in 
fact, it could be made dependent on the past of the identification estimates 
in equation (14) . 


The asymptotically optimal controller designed in parallel of the identi- 
fier is given by the following. The resulting form of the controller consists 
of a cascaded discrete- time Kalman filter and deterministic control gain, both 
of these elements using estimates of the unknown system matrices from an iden- 
tifier. Thus the controller structure attains the following simple form: 

u(k) = K“(k)i(k) (17) 

where K“(k) is the steady-state deterministic optimal gain at time k and 
is obtained by a backward iterative procedure on the discrete-time version of 
the matrix Riccati equation 
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( 18 ) 


K^Ck) = -[B^Ck) W^^^B^(k) + R]"iBo(k) W^_^^A„(k) 

« Aj(k) W^^^Ao(k) + A^(k) + W^^^B^Ck)K.(k) + Q 
i = N-1, N-2,' . . . ; Wjj = Q 

A f ^ 

Estimates of the unknown system matrices from the identifier A^Ck), B^(k) are 
used in the above equations in place of Aq, Bq. At any time k, the station- 
ary value of K^(k) that results by iterating backward with respect to i is 
designated (k) • 

The state estimate x(k) in equation (15) is obtained using a discrete- 
time Kalman filter and again uses estimates of the unknown system matrices in 
place of Aq, Bq. The equations for the filter are then given by 

x(k+l) = A^(k) x(k) + BQ(k)[u(k) + y(k)] + Kp(k+ 1) ^z(k+ 1) 

- H^{A^(k) x(k) + B^(k)[u(k) + y(k)]}^ ; x(0) = 0 (19) 

and 

Kp(k+1) = P(k+llk) HQ'^[HoP(k+ llk)Ho + 

P(k+l|k) = A^(k) P(k|k) A^(k) + 

P(k+l|k+l) = [1 - Kp(k+1)HQ] P(k+l|k), P(o/o) specified 

where 

E[w'^(k)w(j) ] = q^I 
E[v'^(k)v(j) ] = a^l 

A saturation-type nonlinearity is inserted in the feedback loop to induce a 
stabilizing effect on the overall system and to more naturally depict the 
characteristics of all physical feedback controllers (i.e., they inherently 
saturate at a certain level) . 

Simulation of the system in figure 3 using the above controller was per- 
formed on a CDC-6500 digital computer. The results of the identification are 
presented in figure 4 and the performance cost results in figure 5. In these 
figures, the average normalized error represents a 15- run ensemble average and 
hence is an approximation to the mean-square normalized error. It is apparent 
after 10,000 iterations that the identification is consistent. For a sampling 
interval of 0.005 sec, this corresponds to a real time of 50 sec which repre- 
sents about 1/3 of the total flight time. To shorten the identification 
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interval, 0.001 sec/iteration is achievable with present-day general purpose, 
onboard digital computers (for more details, see ref. 24). 


5. PERFORMANCE ADAPTIVE SO CONTROL FOR A SPACE ATTITUDE CONTROL SYSTEM 


The feasibility of application of the expanding subinterval performance 
adaptive SO control method has been investigated in conjunction with a single- 
axis attitude control problem of an orbiting satellite with random switching 
delay (ref. 35). The method was selected because it yields a global minimum 
of accrued performance, it is easy to implement, and does not depend on the 
knowledge of the system or the noise dynamics, thus generating a reliable 
algorithm for control of processes with extremely uncertain dynamics. In the 
particular case of attitude control, the problem may be interpreted as a 
double-^ integral plant with equations of motion: 



= { |u| < 1} 


j(u) 


lim 
t£ °o 




[||x(t)||^ + 0.llu(t)|]dt 


1 


The initial condition x(t£j) was assumed to be zero, and the random variables 
5(t) and n(t) were assumed to be Gaussian with statistics: 


E[5(t)] = 0; cov [5(t); £;(t)] = 6(t-T) 


E[n(t)] = 0; cov [n(t); ti(t)] = 


■c:] 


6(t - t) 


The form of the SO controller is a simplified version of the dead-zone 
controller obtained in reference 35 for a general stochastic fuel regulator 
problem of which this is a special case (see fig. 6) : 


u*(x,t) = -DEZ{cj*[x(t) ,c, t] } 


( 21 ) 


where ai*[x(t) ,c, t] is the predicted value of the approximate switching func- 
tion 0)* [x(t) ,a, t] , included to minimize the influence of the random switching 
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delay with unknown statistics caused by component imperfections. The two 
functions are defined as 
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(23) 


The vector c of the unknown parameters a, $ of dimension m(s-fv) is 
defined as 
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(24) 


The functions ^^[x(t),t], i*l,...,s, (o)*,t) , are properly chosen 

to satisfy the conditions for realizability of the expanding subinterval 
algorithm 


<P(x) i [1, Xj(t), 2x2(t)]^ 
t|)(w) ^ [w(x), w(x)]^ 


(25) 


Consequently, 


c 


[a 


ir 
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^11 


6121 


(26) 


for which the parameter space 
satellites to be 


{c: 0.05 < < 0.5, 

“13 “ “12* ^11 “ 6 


was taken from real data for orbiting 

I“ll/“l 2 l ^ 

2 = 0 } (cases 1 and 2) 


{c: 0.05 < < 0.5, l®xi^®12l - ^ 

‘“13 “ “l 2 ‘ ^11 “ 0 - 0*5} (case 3) 


(27) 


where cases 1 and 2 treat the problem without compensating the random switch- 
ing delay (e.g., to* = w in eqs. (20) and (21)), while case 3 does. 



The random search was implemented by 


'i+1 


( P^ . J(c.) - J(p ,) > 2p ^ 

i at Ti 

Ci* J(c^) - J(p^) < 2y j 


where is the expanding subinterval 

— t “ t^_ i = i,2,...,®* Tj 

over which the following "subgoal" is defined: 


< Tj Vi and lim = «> 

i-Xx) 


( 28 ) 


(29) 




|x(t) I 1 2 + 0.1 |uj (t) |dt 


' 1-1 


(30) 


In equation (28),. is the ith sample value of a vector random variable 
defined at every subinterval on = {c/uc(t)eJ^u}, with probability den- 

sity function p(p) 0 for all peJ2^; y is an arbitrary positive number. 

It has been shown in reference 33 that, if the performance cost accrued during 
the search is defined by 


k 


V(k) = 


2 


)} 


k 


k = 1,2, . . . 


(31) 


1=1 


then in the limit and under certain conditions of convergence (ref. 33), 


lim Prob [V(k) - < 36] = 1 ; 

k-x» 

I(u*) = Min I(u^) = < “ 

c 


(32) 


An EAI-680 analog computer was used in the simulation- As expected, the 
system is very sensitive to changes in the switching delay. In fact, for time 
scaling beyond 1000:1, the computer switching tolerance and d 3 mamic response 
are such that analog simulation is not feasible. On the other hand, the sys- 
tem is quite insensitive to the exact filter gain. To analyze the "transient” 
aspects, three cases of the proposed control were considered. 


For cases 1 and 2, no attempt was made to compensate for the random 
switching delay. The difference in these two cases is in the form of the 
probability density p(p). In case 1, the expected value of the "test-point" 
p(k) is the present value of the parameter vector ot(k), while in case 2 a 
bias is included to reinforce the motion in a successful direction. Case 3 
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is similar to case 2, but it adds the "predictive” aspect to reduce the effect 
of the random switching delay. The experimental results are given in terms of 
an ensemble average in a normalized per iteration cost instead of V(k), as 
defined in equation (29) , because it is more meaningful during the transient 
phase. 

The average per iteration cost and its standard deviations are plotted in 
figures 7 and 8. The statistics depicted in figures 1 and 2 are based on 
sample sets of 100 Independent simulations. Each run was sufficiently long to 
establish the steady-state per iteration cost. In all runs the limiting accu- 
racy was governed by the accuracy of reading the analog records. It is 
observed that both the average and standard deviations of the per iteration 
cost have converged, in a practical sense, within the first 100 iterations. 
Acceleration schemes, based on heuristic selection of the search variable 
in equation (28) have been used in the simulation (ref. 35). Figure 9 demon- 
strates the steady-state responses of the system before and after the tran- 
sient period. 


6. RELIABILITY CONSIDERATIONS 


The reliability problem in the design of feedback controllers for air- 
craft and spacecraft systems may be considered from a different aspect, in 
view of the previous discussion. Modern high-speed vehicles with uncertain- 
ties in their dynamic models operate in a random environment. Stochastic 
solutions (refs. 36 and 37) are based only mostly on average a informa- 

tion about the uncertainties and are usually poor approximations of the 
desired nonlinear and optimal solution (ref. 38). Such solutions are highly., 
unreliable for practical design purposes. Considerable improvement in relia- 
bility may be obtained if the process uncertainties may be reduced through a 
learning scheme in the controller, designed to acquire as 3 nnptotically its 
optimal value. SO controls provide such controllers that are also easy to 
implement and require considerably smaller computers than the general stochas- 
tic solutions. The previously presented feasibility studies should provide 
sufficient arguments to support this statement. From other comparative 
studies (ref. 12), it was found that the parallel identification estimation 
and control algorithm is 6 times faster, requires 18 times less core memory, 
has a 30-percent lower value of the performance cost over a linearized approx- 
imation of the stochastic optimal with unknown parameters, is 3 times slower, 
has the same core memory requirements, and gives 4 times higher performance 
cost than the stochastic optimal with known parameters. The expanding sub- 
interval algorithm designed for systems with almost unknown dynamics is rather 
slower than the parameter adaptive algorithm but compares favorably otherwise 
(ref. 12). Additional acceleration may further speed up the random search 
technique. The above results were obtained from simulations on a CDC-6500 
digital computer. 
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7, DISCUSSION 


In conclusion, improvement in the reliability of the control algorithms 
may be obtained by using learning techniques to reduce the uncertainties of 
the systems and obtain an asymptotically optimal solution in a stochastic 
environment. These ideas have already been explored with various degrees of 
success by various areas of modem large-scale systems, as bioengineering, 
socio-economics, environmental, man-machine systems, etc. It is appropriate 
to mention that learning system approaches are superior to classical and other 
modem control methods only when the system involved is complex, when there 
are uncertainties in the system that can be reduced by learning instead of 
using their average values, and if an optimal solution could be obtained if 
the model of the system was completely known. Then, by use of learning tech- 
niques, one obtains solutions that are more reliable asymptotically than 
otherwise. However, if the system is simple enough that a stochastic or dual 
optimal solution is available, such a method will produce an overdesigned con- 
troller with very poor engineering characteristics. For this reason, certain 
over-designed learning controllers have created some skepticism already about 
learning and SO control to engineering circles. This skepticism, although 
justifiable, should not be extended to the problems considered here. 

The problem of designing a SO control is by no means completed in the two 
feasibility studies presented. Stability analysis safety specification, etc., 
are required for a reliable engineering design. Some of these problems are 
discussed in reference 12. Most of the problems encountered in the design of 
SO control systems are still under investigation. However, the purposes of 
this paper are to motivate interest and to inspire enough confidence in the 
researchers and development engineers in the area to use these ideas in modem 
aircraft and spacecraft technology. 
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Figure 6.- Performance adaptive self-organizing control for the stochastic 

fuel regulator problem. 
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Figure 9*- Steady-state responses before and after self-organization. 





STATUS OF DUAL CONTROL THEORY* 


Edison Tse 

Stanford University, Calif. 
I . INTRODUCTION 


In many processes arising in engineering, economic, and biological sys- 
tems, the problem of decision-making (or control) under various sources of 
uncertainties is inherent. These uncertainties prevent exact determination 
of the effect of all present and future actions, and therefore deterministic 
control theory is not applicable. If the effect of these uncertainties is 
small, one can still use optimal control theory to obtain a feedback control 
law based on deterministic considerations. The feedback nature of the control 
would tend to reduce the sensitivity to uncertainties but would require the 
state of the system to be measured exactly. Again, this assumption is good 
only when the measurement error is small in comparison with the signal being 
measured . 

In many cases, the phenomena of uncertainty (including measurement error) 
can be appropriately modelled as stochastic processes, allowing them to be 
considered via stochastic optimal control theory. A very important concept 
in stochastic control is the 'information pattern available to a controller 
at a specific time, for the purpose of decision-making. As the process 
unfolds, additional information becomes available to the controller. This 
Information may come about accidentally through past control actions or as 
a result of active probing which itself is a possible control policy. Thus 
"learning" is present, whether it is "accidental" or "deliberate." The infor- 
mation pattern available to the controller indicates not only what type of 
learning is possible at each instant of time, but, more importantly, whether 
future learning can be anticipated and how it could be influenced by present 
action (i.e., whether probing would be helpful in future learning). Since 
more learning may improve overall control performance, the probing signal may 
indirectly help in controlling the stochastic system. On the other hand, 
excessive probing should not be allowed even though it may promote learning 
because it is "expensive" in the sense that it will generally increase the 
expected cost performance of the system. This interplay between learning and 
control is the key issue of stochastic control theory. 

This "dual" purpose of the control was pointed out by FelMbaum (ref. 1) 
using the stochastic dynamic programming approach (ref. 2). Because of this 
dual effect, FelMbaum called it "dual control theory." The subject was then 
studied by Aoki (refs. 3 and A), Astrom (ref. 5), Florentin (ref. 6), and 


*Research supported by Office of Naval Research under ONR N0001A-75-C-0738. 
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Meier (ref. 7), via some simple two-stage examples. These studies, however, 
did not display the full force of the dual effects since the control period is 
too short for any extra learning effort to be paid off in the future. To 
fully appreciate the Interplay between learning and control, a N-period'^ con- 
trol problem (N » 3) must be considered. Unfortunately, the optimal dual 
control solution cannot be obtained numerically for N > 3. This motivates 
several recent developments of dual control methods by Tse et al, (refs. 8-10), 
Chow (ref. 11), MacRae (ref. 12), Alster and Belanger (ref. 13), and Alspach 
(ref. 14). The method developed by Tse et at. has been studied to a great 
extent and is applicable to a fairly large class of nonlinear stochastic 
systems, whereas the other methods are developed for specific cases of Inter- 
ests. 


We shall discuss here the recent developments In dual control methods. 

The method proposed by Tse et at. (ref. 8) Is discussed In detail, whereas 
the other methods are only described briefly due to their specific nature. 
Simulation studies are summarized and the future of dual control Is discussed. 


II. DUAL CONTROL: AN ACTIVE CONTROLLER 


To begin our discussions, let us first Introduce the concept of dual 
control. Consider a linear static system given by 

Xr = auj^ + ; k = 1, 2 (1) 


where X|^ Is the observation, a Is the unknown parameter which Is Gaussian 
with mean and variance o^, uj^ Is the control, and Is a sequence 

of equldlstrlbuted and Independent Gaussian variables with mean zero and 
variance q. The stochastic control problem is to find a control law 

Uj “ * “2 “ U2(Xj,Ui,q,a^,0^) 

such that the cost 

J = e|x^ *2 'I “2 ^ 

Is minimized. First, note that since equation (1) Is a static linear system. 
If a Is known, the optimal closed-loop law would be 

uj «= u* = -a (3) 

For unknown a, the certainty equivalence (CE) controller Is 


CE 


u. 


-a 


CE 


u. 


-a. 


( 4 ) 


CE 

where is the updated mean of a after (x^,u^ ) are being observed. 
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Let xx^ be an arbitrary control value applied to equation (1) . After 
the observation of x, , one can compute the a posteriori distribution for a 
which is Gaussian with mean 


uf + q 


(5) 


and variance 


V 

OpUj + q 


( 6 ) 


From equation (6), note that, If |u^| Is large, the updated variance for 

a will be small; In fact. If Ju^l ->• <*> , -»■ 0. This Implies that a large 

control value at k * 1 helps learning of the unknown parameter a. On the 
other hand. If Uj^ “ 0, = Oq and = a , that Is, no additional 

knowledge on a will be provided If a small control value Is used at k ■ 1. 
To see what should be the proper or optimum value for u , one has to calcu- 
late the value, or the ueefutneee, of additional information on a. In the 
context of the given control problem, so that one can trade off present con- 
trol (set u, = -a ) with additional learning for future control (set |u^| 
large). ° 


Let Uj^ be the value of the control at k ■ 1 and be the resulting 
(random) observation. In minimizing the remaining cost to go. 




(7) 


where is given by equation (5), the optimum control law at k = 2 is 


Urt = "Ot = “ 

^2 1 


“pq + 

u2 Op + q 


( 8 ) 


Note that u^ is random since is random. The minimum cost to go: 




, /a q + a u,x. 
1 / o oi l 

u?a + q 
1 o ^ 


(9) 


Is a random variable through x. . The cost of learning Is negative and Is 
represented by the expected minimum cost to go : 
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- /a q + a u,x_ \ 

. -iLa +_Vl_ I 

\ U^a + q J 


This indicates that learning is beneficial (negative cost) to future control. 
The overall cost of applying u^ is 


J = e|xj + 7 Uj} + E{J*(x^,Uj)} 


I = Vi +1^^? 

I “1% + ^ I 

The optimxan value for u^ that will balance between present control 
(minimizing “qUj + (l/2)uj) and additional learning for future control 

(minimizing -(l/2){a^ + (a2u|)/(u^OQ + q)}) is obtained by minimizing the 

"dual cost" given by equation (11) . 

The two components of the cost 

c^(u) = a^u + Y (direct control cost) 


C^(u) = - 2 


(indirect control cost) 


are plotted in figure 1 (where is assumed positive). From the figure, 

it is clear that 

it 

(1) if q is small, u^ - since, in this case, unplanned (or 

accidental) learning is sufficient to reduce the updated variance (eq. (6)). 

(2) if q is moderate, u* is to the left of indicating that 

planned learning does pay off. 

(3) if q is very large, u* == “CXo> since, in the case of high noise 
intensity, additional learning does not pay off and thus the controller might 
as well look at it as a one-stage optimization problem. 

Similar conclusions can be drawn for different range of values for In 

fact, the optimum U| is determined by the ratio 


iCS 



In this static example, the effect of the control value does not 
propagate in time through the system, and thus the effects of learning and 
control can be nicely distinguished. For a dynamic system, control applied 
at a certain time will influence the future state trajectory, and thus the 
dual effects of the control would be difficult to separate. This example 
serves to point out one very important concept of dual control, namely, that 
planned learning should be done only if (1) accidental learning is not 
adequate or (2) planned learning does pay off. This is the central theme of 
dual control theory. 


III. PROBLEM FORMULATION AND OPTIMAL SOLUTION 


In this section, the formulation and solution of the optimal stochastic 
control problem for discrete-time systems are discussed. Consider a discrete- 
time nonlinear system described by 

2^+1 = -k ’ k = 0, 1, N-1 (14) 

and nonlinear observation described by 

-k ’ k = 1, ..., N (15) 

where x is the initial condition, a random variable with mean x i 
and covariance sequences of process 

and measurement noises, respectively, mutually independent, white and with 
known statistics. For simplicity, they may be assumed to have zero mean. 
Consider further the performance measure 


N-1 

|k(2En> + (16) 


where the expectation E{*} is taken over all underlying random quantities. 
Finally, consider admissible control of the closed-loop type: 

Ufe = •••» -k-1^ 

* N-1 

The goal is to find the optimal control law which has the form of 

equation (17) and minimizes the cost (eq. (16)) subject to the constraints 
(eqs. (14) and (15)). 


Suppose that a certain control law ^(*) is chosen for k = 0, 1, ..., 
N-2, and the observation sequence is observed. The remaining problem 
is to select , so that the conditional expected cost 
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N-2 

-E 

. k=0 


(18)1 


Is minimized. Since the summation term Is Independent of 2 * optimum 
u^ ^ Is obtained by minimizing 

E{Ljj_j (Xn_i ) + ^(3%) I 

+ dv^_j 

Note that p(v,^ i) is known a prioxn., and thus the only on-line quantity 

N-i -N-2 

needed in the calculation of equation (19) is either (Y ,U ) or 
p( 2 ^ ^); thus the optimal control must be of the form 

uj.! = or uj_^ = (20) 

Substitute equation (20) Into (19) and define the optimal cost to go; 

( 21 ) 

where iPn-i Is the information state that can either be represented by 

(yN-l^uN-2) p(xn-iI^^ 1,U^ ^) . The cost of using the control law 
N— 2 

{l^(*)}j^_p, with the knowledge that the remaining control law is optimal, is 




(22) 


iWhen a control law is specified, notation u.^ is used to denote the 


"*kA ■“ “ 

realization of the control value at step 1; U ^u , ..., uu } 
realized. 


when Y is 
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To carry this procedure backward in time, assume that is chosen, 

k « 0, 1, ..., N-3, and is observed. If a particular control value 

— N-2 chosen, then, from equations (14) and (15), a dynamic description of 

the information state is available: 

’^N-l “ (23) 


where „ is the information state at N-2 and Y„ , is the random 
N-2 N-1 

observation to be observed at time N-1. If, for example, the conditional 

density for the current state is used as an information state, then 


♦n- 1 ■ P(SN-ll**'''-"*'‘^JiN-2’ -■^P(2N-ll2N-l)yP0%_il*K.2.'^.2)x' 

, |„N-2 r,N-3. , 

P(%_2|Y .U )dXj^_2 

“ \ P V 2‘^%-2 


(24) 


The optimal cost to go is given by 


^N-2^'^N-2^ ^ i ^N-2^%-2’%-2^ ^N-1^’^N-1 ^’*'n- 2'%-2’^N-1^^ l%-2 f 


■^-2 


and the cost of using the control law 
remaining control law is optimal, is 


(u^O) 


N-3 

k=0 


(25) 

with the knowledge that the 



V2'*N-2> 


(26) 


Inductively, the conditional expected optimal cost to go, satisfies 

the dynamic programming equation: 


Ik(V = “in E{Lj^(Xj^,u^^) + (^7) 

with end condition 

In(V = E{K(x^)|y (28) 

The optimal control law is obtained by solving equations (27) and (28). 

It is well know that the solution of equations (27) and (28) is feasible 
only for a small class of problems (refs. 3 and 8). In most cases, one is 
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unable to solve equations (27) and (28) due to the curse of dimensionality 
(ref. 2), and thus suboptlmal procedures must be developed for the problem. 
However* one can deduce the dual nature of the optimal control from the 
dynamic programming equation (27) . Depending on the actual realization of 
the overall cost of applying the control ^ at time k consists of 
two components: direct cost, E{Ly(x ] , ,u y ) 1 > and Indirect cost, 

] I . The direct cost represents the conditional 

expected cost that would have been Incurred If the problem Is a one-shot 
static problem, whereas the Indirect cost represents the conditional expected 
cost of the future information state, which depends on U|^ because of the 
dynamic description of the system equation (14) and the observation relation 
equation (15) . The dual effects of are Interrelated and embedded In 

the Indirect cost component. 


IV. A DUAL CONTROL METHOD 


In this section, we shall describe a dual control method due to Tse 
et at. (refs. 8 and 9). Note that the main difficulty In solving equation (27) 
hinges on the calculation of the indirect cost ] 1 * 

If, however, this quantity can be obtained exactly or approximately, then 
at every time k, it looks like we have a two-stage stochastic control 
problem. With this motivation, Tse et at. propose to approximate equation (27) 
as follows: 


(1) Approximate the Information state so that the dimension of the 

Information state space stays constant In time. It Is suggested that the 
conditional mean and covariance be used as the approximated Information state. 
Denote such an approximated information state by I I ^ 


(2) Approximate the optimal cost-to-go function associated with an 
approximated information state by performing a second-order perturbation 
analysis around a nominal trajectory that Is different for different predicted 
approximated information states. Denote the approximation for the optimal to 
go by which is given by (see refs. 8 and 15 for 

details) 


- ■’d'" - + •’P™ - <29) 

where 

N-l 

Jjj(N - k) = K(x^) + 53 Lj( 2 cJ,Uj) + Y^(k + 1) + \(4;|k 

j — k“H 1 

Is the deterministic part of the cost along the nominal 


^We denote the conditional error covariance by either or ^(k[k). 
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, j - k + 1 N-1 ; - E{xj^+ll\.Uk> 


and 


J„(N - k) =i tr[K (k + l)l(k + l|k)] + 4 E tr[K (j + l)^(j)] 


N-1 


j-k+1 


N-1 

tr [j 

j.ir+1 ‘>•32 


Jp(N- W 


are the stochastic terms In the cost evaluated along the nominal. The 

matrices K (J),4 (j) and the scalar y (j) are given by 

— o o.xx o 


with 


Y (j) “ Y (j + 1) - 4 H’ (j)2ff ^ (j)H (j) 
'o 'o 2 o,u o,uu ■' o.u'-^' 


j=N-l k + 1; Yjj(N) “ 0 

P (j) “ H (j) - ?C' (j)H (j) 

-o o,x^-^'^ o,^ o,uu'-^' o,u 

j-N-1, ...,k+l; 2 o<N) = 




J-K-1 k + lj K (S) - i|> 


O.XX 


H 


,(j) 


} 

} 

} 


^ (j) =H (j) + f (j)K (j + l)f (j) 

o,22L o,xx — o,x “O --o,x 


(j) = H (j) + f* (j)K (j + l)f (j) 


K = H _(j) + f: „(j)K (j + i)f (j) 


o,uu 


o>uu 


O.U “ 


.U 


A (j) = 5C (j)3C (j) 

o,xx o,ux o,uu'-'‘^ o.ux 


L^Cj) = Lj(Xj,Uj),f^(j) =fj(Xj.Uj). 


(31) 

(32) 

(33) 

(34) 

(35) 

(36) ^ 

(37) 

(38) 

(39) 

(40) 
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( 41 ) 


(3) At each time k, an on-line optimization problem 


min 


Ui 

— k 


DC^- 


is solved in real time. Denote the minimizing by u. (^j.) • Apply 

('Pj^)s update when is observed* and resolve the problem for 


k + 1. 


One distinguishing characteristic of this approach is that it explicitly 
takes into account that future measurements will be made, and these future - 
measurements will provide additional information about the uncertain param- 
eters that will enhance future control. Therefore, learning is planned in 
accordance with the control objective. This gives the control law a closed- 
loop characteristic (see refs. 15 and 16). To see this, first note that 
Y(j(k + 1), £(k + 1), and ^(k + 1) are not stochastic and depend only on the 
nominal control sequence being chosen. The function Hq(J) defined by 
equation (36) is the Hamiltonian for the corresponding deterministic optimal 
control problem. Therefore, if the nominal control is a local minimum for 
the deteministic problem, then Ho^u(j) = ^ and thus Yo(J) “ also 

is positive definite for all j > k + 1. The latter implies that 
the“~perturbation analysis is valid. In fact, a necessary and sufficient 
condition for the existence and uniqueness of the optimum perturbation solu- 
tion is that ^,uu(j). niust be positive definite (regardless of whether the 
nominal control is“a local minimum). If ^^uu(J^ positive definite, 
then >5o,xx(J) (defined by eq. (40)) must be nonnegative definite, with rank 

equal to the dimension of the control vector. Now observe that the first 
stochastic term (eq. (31)) reflects the effect of the uncertainty at time k 
and subsequent process noises on the cost. These uncertainties cannot be 
affected by ^(k) , but their weightings depend on it. The effect of these 
uncontrollable uncertainties on the cost should be minimized by the control; 
this term indicates the need for the control to be cautious and thus is called 
the caution term. The second stochastic term, equation (32) , accounts for the 
effect of micertaintles when subsequent decisions will be made. As discussed 
above, if the perturbation problem has a solution, then the weighting of 
these future uncertainties is nonnegative (Aq is positive semldefinite) . 

If the control can reduce by probing (experimen^tion) the future updated 
covariances, it can thus reduce the cost. The weighting matrix Aq 
yields approximately the value of future information for the problemTjiider 
consideration. Therefore, this is called the probing term. Since the rank of 
Af, xx(j) equal to the dimension of the control vector, this indicates that, 
along a certain nominal sequence, certain combinations of state uncertainties 
do not contribute (up to second order) to the total average cost. 


The benefit of probing is weighted by its cost and a compromise is chosen 
to minimize the sum of the deterministic, caution, and probing terms. The 
minimization will also achieve a tradeoff between the present and future 
actions according to the Information available when the corresponding deci- 
sions are made. 
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Figure 2 is a flow chart of the algorithm. 


V. OTHER DUAL CONTROL METHODS 


Several other dual control methods are suggested in the literature. But 
almost all are developed for limited classes of problems. Chow (ref. 11) and 
MacRae (ref. 12) developed two different dual control methods mainly for the 
control of linear econometric models with unknown parameters. Chow’s method 
is based on an approximation of the optimal cost to go. In this respect, his 
algorithm is similar to the one described in section IV. There are some basic 
differences between Chow’s method and the one developed by Tse et aZ. One is 
that Chow employed a single nominal whereas Tse' et al. allowed to have multi- 
nominals; another difference is that Chow proposed a step-by-step numerical 
approximation of the optimal cost to go, whereas Tse et at. used perturbation 
analysis to obtain the optimal cost to go. MacRae’ s approach is also based 
on dynamic programming, but whenever a future updated covariance appeared in 
the equation, she replaced it by a first-order approximation about a nominal 
trajectory. 

Alster and Belanger (ref. 13) developed an ad-hoc dual control method 
for the control of a class of linear discrete time systems with unknown 
constant parameters. They first rewrite the expected cost conditioned to the 
current time period, and suggested to minimize the open-loop expected cost 
subject to an added constraint that the next period’s updated covariance is 
smaller than some threshold. While this is a plausible approach to the prob- 
lem, the problem of selecting the threshold is nontrivial and probably highly 
problem dependent. Kalaba and Detchmendy (ref. 17) suggested a similar 
approach where the original cost is modified to include penalty of having bad 
estimates. Alspach (ref. 14) used perturbation methods to develop a dual con- 
trol method for the control of linear systems with known dynamics but with 
multiplicative noise. 

All methods described in this section assumed a quadratic criterion. 


VI. SIMULATION STUDIES 


Up until now, no simulation study using the methods proposed by Chow, 
MacRae, and Kalaba and Detchmendy has been reported in the literature. 

Limited simulation studies were reported in reference 13 where their algorithm 
is compared to the certainty equivalence (CE) control algorithm for the control 
of single-input/single-output linear systems with unknown parameters under 
the minimum output variance criterion. Alspach (ref. 14) also compared his 
algorithm with the CE algorithm for the control of single- input/single-output 
linear systems with multiplicative noise under the quadratic criterion. In 
both studies, the proposed dual control methods perform better than the CE 
method. 
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Tse et at. applied their dual control methods to several representative 
cases in three different classes of problems. In each case, the dual control 
method is an order of magnitude better than the CE method. Moreover, the 
similatioh studies revealed several important characteristics of dual control. 
The first class of problems is the control of linear systems with unknown 
parameters. Tse et al. (refs. 8 and 9) studied the control of single-input/ 
single-output linear systems with unknown parameters under the quadratic 
criterion. By varying the weighting matrices, the active learning character- 
istics of the dual control is vividly displayed. The second class of problems 
is that of controlling nonlinear systems. For such a class of problems, 
there is no unknown parameter to be learned. However, there is another 
important characteristic of the dual control: the capability of bringing the 

system state to a certain region that will improve state estimation. This is 
clearly shown in the studies by Tse and Bar- Shalom (ref. 10), where they 
applied their dual control method to an interception problem. The third class 
is the control of dynamic systems with drifting parameters. Pekelman and Tse 
(ref. 18) used the dual control method in the problem of optimum decision on 
advertisement expenditure. In this case, the system is nonlinear with 
drifting (unknown) parameters. This study also indicates that when the 
knowledge of the drifting parameters is crucial to the control performance, 
the dual control method will induce enough learning to track the variation of 
the parameters . 


VII. CONCLUDING REMARKS AND THE FUTURE OF DUAL CONTROL THEORY 


Generally, it is found that dual control methods give a better perfor- 
mance, but they are also considerably more complicated than the CE control 
method. The question then is how much we want to trade off between perfor- 
mance improvement and complexity. In some cases, it can be shown that the 
CE method gives reasonably good results (ref. 19) or even the optimal solu- 
tion (refs. 16, 20, and 21), and thus the dual control is not needed. Clearly, 
one important question to be answered in any stochastic control problem is 
how much we can expect to gain by using dual control instead of CE control. 
Theoretical studies in this direction are not found in the literature. From 
personal experience, a problem with the following characteristics may need 
dual control approach: 

(1) The control period is relatively short, but not too short, so that 
the benefit of learning does not contribute substantially to the indirect 
cost. 

(2) State estimation performance is highly dependent on the state 
trajectory. 

(3) Control performance is highly dependent on the knowledge of the 
unknown parameters in the system. 

(4) The natural tendency of the system is incompatible with the system 
performance, for example, regulating the output of an unstable system close 
to zero or steering the output of a stable system to a specified value. 
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To understand the similarities and differences of all the proposed dual 
control methods, comparison studies of these methods should be carried out 
using the same data base. Bar-Shalom and Tse (ref. 22) initiated effort in 
this direction by comparing the methods of Tse et at* (ref. 8), Chow (ref. 11), 
and MacRae (ref. 12) via a simple three-stage linear system with unknown feed- 
forward gain parameter. Comparison studies for more complicated problems will 
be reported in the future. 

Throughout the discussions, it was assumed that the structural form of the 
system dynamic and the observation relation are known to the controller. 

While this assumption is valid in many engineering applications, it may not be 
valid in many processes arising in economic, chemical, and biological systems 
where a unique description of the system structure is impossible. A formula- 
tion of combined system identification and control for systems with ambiguity 
in structural description seems to be needed. 

One possible formulation is to assume that the underlying system can 
have M different possible structural descriptions. These descriptions 
may be derived from several equally possible hypothetical assumptions. The 
problem then is that of minimizing a certain cost criterion, subject to the 
constraint that the true dynamic equation can be one of the M possible 
difference equations. This problem has been considered in references 23 
through 26. These approaches are equivalent to separation of detection and 
control. While the methods seem to be useful in some applications (ref. 27), 
in some cases, the method faces the problem of identif lability where the 
”true” system cannot be identified even at k (For a discussion of 

identif lability problems, see refs. 28 to 32.) An active approach similar 
to the dual control (as described in section VI) would be useful. In this 
case, the control would have the following "dual" purposes. 

(1) To allow resolvability of model structures and to help in estimating 
the unknown parameters of a given model structure. 

(2) To regulate or control the system to follow some desired path or 
performance measure. 

For M different possible model structure descriptions, the problem 
should be formulated as a simultaneous hypothesis testing and control problem. 
Such an active approach has not yet been attempted. 
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CLOSED-LOOP STRUCTURAL STABILITY FOR 
LINEAR-QUADRATIC OPTIMAL SYSTEMS* 

Poh Kam Wong and Michael Athans 
Massachusetts Institute of Technology 


SUMMARY 


This paper contains an explicit parametrization of a subclass of linear 
constant gain feedback maps that never destabilize an originally open-loop 
stable system. These results can then be used to obtain several new struc- 
tural stability results for multiinput linear-quadratic feedback optimal 
designs. 


I, INTRODUCTION AND MOTIVATION 


This paper presents preliminary results which, in our opinion, represent 
a first necessary step in the systematic computer-aided design of reliable 
control systems for future aircraft. It is widely recognized that advances in 
active control aircraft and control configured vehicles will require the auto- 
matic control of several actuators so that future aircraft can be flown char- 
acterized by reduced stability margins and additional flexure modes. 

As a starting point for our motivation, we must postulate that the design 
of future stability augmentation systems must be a multivariable design prob- 
lem. As such, traditional single- input /single-output system design tools 
based on classical control theory cannot be effectively used, especially in a 
computer-aided design context. Since modern control theory provides a concep- 
tual theoretical and algorithmic tool for design, especially in the linear- 
quadratic-Gaussian (LQG) context (see, e.g, , ref. 1), it deserves a special 
look as a starting point in the investigation. 

In spite of the tremendous explosion of reported results in LQG multi- 
variable design, the robustness properties have been neglected. Experience 
has shown that LQG designs "work’’ very well if the mathematical models upon 
which the design is based are somewhat accurate. There are several sensitiv- 
ity studies involving "small-parameter perturbations" associated with the LQG 
problem. We submit, however, that the general problem of sensitivity and even 
stability of multivariable LQG designs under Zapge ipoTometv%Q and structurat 
changes is an open research area. 


*This research was conducted in the M.I.T. Electronic Systems Laboratory 
with partial support extended by NASA Ames Research Center under grant 
NGL-22-009-124 and by the Air Force Office of Scientific Research under 
grant AF-AFOSR-72-2273. 
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It is useful to reflect on the basic methodology in classical servo- - ^ 

mechanism theory which dealt with such large-parameter changes • The overall K 
sensitivity and stability considerations were captured in the definition of , . 
gain and phase margins. If a closed-loop system were characterized by reason- 
able gain and phase margins, then (a) reasonable changes in the parameters of 
the open-loop transfer functions and (b) changes in the loop gains due, for 
example, to saturation and other nonlinearities could be accommodated with 
guaranteed stability and at the price of somewhat degraded performance. 

Although LQG designs are time-domain oriented, nonetheless their 
frequency-domain interpretations are important, although not universally 
appreciated. For example, for the single- input/single-output linear-quadratic 
(LQ) optimal designs, Anderson and Moore (ref. 2) have shown that LQ-optimal , 
designs are characterized by (a) an infinite gain margin property and (b) a . 
phase margin of at least 60®. Such results are valuable because it can be 
readily appreciated that, at least in the single-input/single-output case, 
modern control theory designs tend to have a good degree of robustness, as 
measured by the classical criteria of gain and phase margin. 

Advances in the multiinput /multioutput case, however, have been scattered 
and certainly have not arrived at the cookbook design stage. Multivariable 
system design is extremely complex.^ To a certain extent, the nvmevidal sotu~^ 
tion of LQ optimal is very easy. However, fundamental understanding of the i 
structural interdependencies and their interactions with the weighting 
matrices is not a trivial matter. We believe that such fundamental under- 
standing is crucial for robust designs as well as for reliable designs that 
involve a certain degree of redundancy in controls and sensors. 

The recent S.M. thesis by Wong (ref. 3) represents a preliminary yet 
positive contribution in this area. In fact, the technical portion of this 
paper represents a slight modification of some of the results reported in 
reference 3. In particular, we focus our attention on the stability proper- 
ties of closed-loop systems designed on the basis of LQ-optimal techniques 
when the system matrices and loop gains undergo large variations. 

The main contributions reported here are the eventual results of gener- 
alizing the concepts of gain margin and of performing large-perturbation sen- 
sitivity analysis for multivariable linear systems designed via the LQ 
approach. We warn the reader that much additional theoretical and applied 
research is needed before the implications of these theoretical results can 
(a) be fully understood and (b) translated into systematic "cookbook” pro- 
cedures that have the same value as the conventional results in classical 
servomechanism design. 

This paper is organized as follows: in section 2 we present an explicit^ 

parametrization of a subclass of linear constant feedback maps that never 
destabilize an originally open-loop stable system and establish some of its 
properties. In section 3, we apply this construct to obtain several new 


^Even the notion of what constitutes a "zero” of a multivariable transfer 
matrix was not fully appreciated until recently. 
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closed-iloop structural stability characterizations of multiinput LQ-optimal 
feedback maps. We conclude in section 4 with a brief discussion of the rele- 
vance of the present results for computer-aided iterative feedback design - 

Notation 

(1) The linear time- invariant system 

^(t) = _^(t) + Bu(t) 
z(t) =HTx(t) 

where 

x(t) £ 3t(*) = state vector 

u(t) £ i?™ u(*) = control vector 

^(t) E i? ^(*) = output vector 

and A E _B E and E will be denoted by 2] (A,j3,Il^) . Where 

is irrelevant to the discussion, we shorten the notation to ^(A,JB) and, 
where the choice A,^ is clear from the context, we simply use Z. 

If the matrix A is stable (i.e., all eigenvalues of A have strictly 
negative real parts), we refer to ^ stable system. 

(2) R(K) is range space of K, N(K) is null space (kernel) of K, and 
Rk(K) is rank of K. 

(3) Given system ZCA,^,!!^), 

R(A,A) = controllable subspace of the pair (A,JB) 

= R(B) + ^(B) + ... + A*^"^R(B) 

T A T* 

N(H ,A) = unobservable subspace of the pair (H'^,A) 

= n n(h'^a^“^) 
i=l 

(4) If is positive semidefinite , we write ^ If ^ is 

positive definite, we write ^ > 0.- 
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II. PARAMETRIZATION OF NONDESTABILIZING FEEDBACK MAPS 


We begin our discussion as follows. 


Definition 1 

Given the stable system Z (A,]B) , let 

S(Z) = {g"^ e (a + Bg"^) is stable} 

that is, S(Z) is the set of all feedback maps t^at never destabilize an 
originally open- loop stable system, where _u(t) = * 

Ideally, one would like to be able to explicitly parametrize S(E), but 
as this is a well-known intractable problem, our strategy here is to look for 
a simple parametrlzation of a (hopefully) sufficiently general subset of 
S(I). 


We begin by first recalling some standard Lyapunov-type results: 

Lemma 1 (Wonham, ref. 4): 

(i) If A . is stable, then the Lyapunov equation 

PA + + ^ = 0 

with 5, 2 ^ has a unique solution ^ If, in addition, > A) is 

observable, then P. > ^. 

(ii) If 

(1) P > 0, ^ > 0 satisfy m + a'^P + ^ = 0 

(2) (^^/^,A) is detectable 
Then A is stable. 

(iii) If ^ and (q| , A) , is obse rvable (detec table) . then for all 

P > ^, R > ^ and for all B,^, the pair (/Q P + FRf'^', A + is 

observable (detectable) . 

Proof: 

For (i) , see reference 4, page 298. 

, .For (ii), see reference 4, page 299. 

For (iii), see reference 4, page 82. 
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To proceed, the following definition will be useful. 


Definition 2 

For any stable A, let 

LP(A) = { K > 0|^ + A% < a} 

LP"’'(A) = {K > Ol^ + A% < 0} 

Remark: LP(A) Is generally a proper subset of the set of all posltlve- 

semldeflnlte matrices of dimension n. 

Example: ■ 

Suppose that ' 


< 0 


Then 


LP(A) = 


pi 

A = , Xj < 0, 


Note that 


(Xi + Xp2 


Ki 

S 0 

fl2 


iff Kj > 0, > 0, KjK2 > K^2 


^ >1, with equality iff = X^- 


Lemma 2: 

(i) LP(A) is a convex cone; that is, Kj, K£ ^ implies 

^1^1 ^ ^2^2 ^ for all > 0, «2 > 0. 

(ii) K eLP(Ai) n LPCA^) implies K G LPCA^ + A^) . 

(Ill) K € LP(A) Implies K G LP(A + B(S - L)b'*’P . 


Proof: 


Straightforward. We are now ready to Introduce our first crucial result 
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Lemma 3: 


Let A be stable. 

Then [A + (N - M)K] Is stable for all K e LP(A) and for all M > 

N = -nT and where R(N) C R(M) . 

If Ke LP"^(A), then the condition R(N) C R(M) can be omitted. 

Proof: 

Let £ = ~(^ + A%). Since K e LP(A), we have ^ > 0, and A stable 
Implies A) always detectable. 

Now ^ + A% + £ = 0, 

so K[A + (N - M)K] + [ A + (N - M)K]\ + 2KMK + ^ - (KNK + - £ 

but = 0 since N « -N^. 

If K E LP'*’(A) , then > 0 

so (/Q + 2KMK , [A + (N - M)K] is observable, which Implies 
[ A + (N - M)K] is stable by Lemma 1 (11). Otherwise, assume R(N) C R(M) , 
which Implies that there exists V such that ]N = TO! or that 
(N - M)K » ( V - 1)»K. 

By defining 1 = ( V - I.)m1/^ 

f ^ m1/2k 



in Lemma 1 (ill), [/Q + 210^ , A + (N - M)K] is detectable. By Lemma 1 (11), 
we therefore have [A + (N - M)K] stable. 


Remark : 

A special case of Lemma 3 was e'stabllshed by Barnett and Storey (ref. 5) 
By specializing Lemma 3, we Immediately obtain an explicit parametrlzatlon of 
a subclass of stabilizing feedback. First we Introduce the following. 
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Definition 3 


Given the stable system £ (A, J() , let 

Sj(£) = {G^ e - (£ - 1 “ -s^» L. ^ 0» 

and either K G LP'*"(A) or else 
kelp (A) with R(S) C R(L)} 

We can now state our result as 
Theorem 1 - 

e 

Given the stable system £(A, ^) , then 

(I) G^ e Sj(E) Implies (A + BG^) Is stable. 

(II) f* 2 e4‘ dt > f 2 dt 

•^0 *^0 

where £ > 0 Is such that ^ + A% + ^ = 0 and G^ = (^ - L)B% G Sj(E) 
Proof: 

(i) Let M = BLb' ^, N = BSB* ^ in Lemma 3, and the result follows 
directly. 

(ii) Let ^ ^ be such that 


Then we have 


M + + ^ = 0. 

^ f“ A^t . At _ 
K = I e- ^ e dt 

•'o 


Next rewrite (*) as 

K(A + BG'^) + ( a + + ( 2KBLB% + = 0 

where G^^ - iS - L)B% G Sj(E) which Implies 

K = f ^^^^( 2KBLB% + 

Jn 


dt 
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hence 


or 


r 2 et . r 


2 e=' dt - I 2 dt 

“'o 


2 r SLB% dt 

Jn 




2 e4‘ dt > r " -<A+M’’)t 

Jr\ 


£ e'- — dt 


QED 


Remark: It can be easily shown that all eigenvalues of the feedback term 

have nonpositive real parts (term -BLB' ^K has only real eigen- 
values, while BSB’ ^K has only pure Imaginary (conjugate pairs) eigenvalues or 
zero eigenvalues). This observation and the content of theorem 1(11) makes It 
convenient to Interpret S^(E) as a natural generalization of the concept of 
"negative" feedback to the multivariable and multllnput case. 


The next two corollaries are easy consequences of theorem 1. , 
Corollary 1.1 - 

Let I (A, J|) be a system with a single Input, that Is, let jB be a 
column (nxl) vector If ..., e Sj(S(A, ^)), then 

A T 

2j “-iSi ^ Sj(E) for all a, > 0, 1 = 1, . . . , j 

1=1 


Proof: 


Each ^ Is of the form r^^Jb^K^ for some admissible r^^, K^, so 

S - 2 “I'A - 

j 

But, from Lemma 2(1), ^ G LP(A) Implies ^ ^ LP(A) for all 


o^r^ > 0; hence 2 “i-S^ ^ LP(A) for all > 0. 


QED 
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Corollary 1.2 - 


Suppose there exists L ^ ^ such that BLB* ^ € LP(a''*’). Then 
[A - BLBT(K + N)] is stable for all K i 0 and N = such that 

R(K) => R(N). 

If BLB^ e LP'*’(A^) actually, then the condition R(K) ^ R(N) can be 
omitted. 

Proof: 

Immediate from "taking the transpose" in Lemma 3. 

QED 

Theorem 1 has illustrated the Importance of LP(A). It is therefore use- 
ful to have an alternative characterization of LP(A): 

Proposition 1 - s ; ' a -:; 

. LP(A)i is A^- invariant, that is, for all K e LP(A) -- i ; : r | 

A%(K) C R(K) : ‘ . ' . 

Proof: 

K G LP(A) iff ISA + A% + Hh'^ = 0 .for some H. 

We claim that N(K) = N(H^, A) = unobservable subspace of (H^> A). 

For K = f e^^^ HH'*^ e^^ dt so x e A) implies e^^ x = 0 

•'o 

for all t e R, which implies x e N(K). Conversely, 3c e N(K) implies 

J ^OO 

I 1®^ x|^ dt = 0 or e^^ x = 0 for all t G R» that is, 

0 

X G N(ir, A). . 

To complete the proof, note that 
R(K) = RCk’^) = 

= NCH^, A)-*- 
= R(A^, H) 

rn 

= controllable subspace of (A^ , H) 

• ’ ■■■* i , ■ . c 

But any controllable subspace of Is necessarily an A^- Invariant subspace. 

QED 
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Remark: The significance of proposition 1 is that it provides a systematic 

means for generating all members of LP(A). For example, if A has ^Istlnct 
real eigenvalues, then every K G LP(A) is of the form 

K = p'^MP 

where the rows of ^ are left eigenvectors of A, that is, 

PA = Ji = diagonal (Aj, ..., 

and M = diagonal m^) , m^>0, i“l, ...jn. Thus, all members of 

LP(A) can be trivially generated once jP is known. 

o 

While membership in S^(Z) is sufficient to guarantee closed-loop sta- 
bility, it is, of course, not necessary, that is, Sj(E) is a strictly proper 
subset of S(Z). Intuitively, if the open-loop system is stable enough to 
begin with, it can tolerate a certain amount of "positive" feedback without 
leading to closed-loop instability. In other words, the poles of the open- 
loop system can be shifted to the right by feedback without destroying stabil- 
ity oSo long as none of them are shifted into the closed right-half plane. By 
allowing such additional nondestabilizing feedback, therefore, we ought to be 
able to "enlarge" Sj(Z). More precisely, we have the following. 

Definition 4 

Given the stable system Z(A, B) and any L ^ L G let 

LP(Z, L) = {K > O]^ + A% + 2KBLb\ < 0} 

LP+(E, L) = {K > 0|^ + A^K + 2KBLb' ^K < 0} 


Definition 5 

Given the stable system 2 (A, B) ^ let 

S^CZ) = {G^ € = (L + 1)b\, £ = 

^ L > ^, and either K e LP‘^(Z, L) or else 

K € LP(Z, L) with R(L + ^) C r(l - T)}. 


Theorem 2 - 

_ T 

Given the stable system I (A, ) , then G^ E 82 ( 2 ^) implies (A + BG ) 

is stable. 


Proof: 


The proof follows by a straightforward extension of the proof of Lemma 3 
and theorem 1 and hence is omitted. 


QED 
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Remark: It can be easily seen that theorem 1 is just a special 
theorem '2 (with L = ;0 and t < jO, S^C^) will be reduced to 
that, in the general case covered by theorem 2, no definiteness 
made of L, and thus various ''mixtures” of positive and negativ 
allowed. 

The next proposition provides further clarification on our 
tion scheme. First define 

Fi(B) = {G^ E D E arbitrary, 

K E i?"™ and K > 0 } 

F2(B) = {G^ E i?“™|RK(G^B) < RK(G^)}. 

Proposition 2 

Fj(B) n F2(B) = <p 

Fj(B) U F2(B) = 

T" mxn 

that is, any feedback map ^ e i? is either in the set F..( 
F2(B). 

Proof: 

We need only to show that 

Fj^(B) = {G^ E i?™™|RK(G^B) = RK(G^)} 

Necessity: Suppose E Fj (B) , that is, there exists D E 

K Ei?^™, K >0 such that 6^“= Then 

T T T T 

G = DB KBD > 0 

SO RK(G^B) > RKCg'^^T) = RK(DB^KBD^) = RK(m'^K) = RK(G^) . 
Sufficiency: 

Take ^ and observe that == G^Bb'^K has a soluti 

RK(G^B) = RK(g''>T. 


We now relate the content of proposition 2 to theorem 2 . 
that 82(2) C Fj and hence our parametrization scheme fails tt 
nondestabilizing feedback map g ^2^— That S(E) n F2(jB) ^ (f> 
strated by the following trivial example: 


133 


Example: 



Note, however, that if B is of full rank, then the set ‘ F2 (B) ±s not ^ 
generic in The more interesting question — Is 22(2) genetic (i.e. , ' " 

dense) in S(Z) n F^(^)? — is presently unsolved. 


Our results so far have been on systems 2 (A, ^) , which are open-loop 
stable; the question next arises as to what the situation would be for systems 
not open-loop stable (i.e., A has unstable poles). For A unstable, it 
is, of course, not possible to write Lyapunov- type equations. One is reminded, 
however, of the algebraic Riccati equations; indeed, we have the following 
interpretation of the traditional LQ-optimization problem: ' . . 


Definition 6 


Given (A, _B) , a stabilizable pair, let 

A T* 

LQ(A, ^) ” {K > ^|k = K(A, Rj H ) for some ^ ^ and some 

T T 

H so that (H , A) is a detectable pair} 

where K(A, B, R, H*^) denotes the unique positive semidefinite solution to 
the algebraic Riccati equation: 

KA + a'^K - KBR"^b\ + = 0 


For R fixed, we denote the corresponding set as LQ(A, R) . 


Definition 7 

83(1:) = e = -R'^b'^K, R > 0 , K € LQ(A, B; R) } 

Proposition 3 - 

T . . . ' T 

Given any stabilizable system 2 (A, ^ E ^3^^) implies (A + ) 

is stable. 

Remark: The above proposition merely summarizes the well-known "standard** 

results of LQ-optimal feedback theory (refs. 1 and 4 ). However, the interpre- 
tation here of the LQ-optimal feedback class (S^C^)) as a parametrization of a 
subclass of stabilizing feedback is interesting. 
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Remark; . It can be easily seen that theorem 1 is just a special case of 
theorem ‘2 (with L = ^ and t < jO, ^ 2 ^^^ will be reduced to S^(I)). Note 
that) in the general case covered by theorem 2 , no definiteness assumption is 
made of L, and thus various "mixtures” of positive and negative feedbacks are 
allowed. 

The next proposition provides further clarification on our parametriza- 
tion scheme. First define 

e = Db\, D € arbitrary, 

K € and K > 0} 

■ ^2^ = {g'^ G i?“™|RK(G'^B) < RKCg"^)}. 

Proposition 2 

Fi(B) n F 2 (B) =4. 

Fj(B) U F2(B) = 
rji mxn 

that is, any feedback map ^ G i? is either in the set F,(jB) or else 

F2(B). 


Proof: 


We need only to show that 

Fi(B) = {G^ G /?™^1 rK(g'^B) = RKCg"^)} 

Necessity: Suppose G Fj^(B), that is, there exists ^ G i?™™ and 

K K>0 such that 6^“= DbTk. Then 

XT T T 

G BD = DB > 0 

SO RK(g’^B) > RK(G^BD'^) = RKCDB'^KBd'^) = RK(DB%) = RK(G^) . 

Sufficiency: 

Take D = G^J3 and observe that ^ = g'^Bb'^K has a solution K > ^ if 

RK(g'^B) = RK(g'JT. 

QED 


We now relate the content of proposition 2 to theorem 2. Observe first 
that 82 ( 2 ) C Fj and hence our parametrization scheme fails to capture any 
nondestabilizing feedback map g ^ 2 (^). That S(E) n F 2 (^) ^ (p is demon- 
strated by the following trivial example: 
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Note, however, that if jB is of full rank, then the set F^CB) is 
generic in The more interesting question — Is ^2^^^ generic (i. e., ’ " 

dense) in S(Z) n — is presently unsolved. 

Our results so. far have been on systems Z(A, ) , which are opien-loop 
stable; the question next arises as to what the situation would be for systems 
not open-loop stable (i.e., A has unstable poles). For A unstable, it 
is, of course, not possible to write Lyapunov- type equations. One is reminded, 
however, of the algebraic Riccati equations; indeed, we have the following 
interpretation of the traditional LQ-optimization problem: ' . 

Definition 6 

Given (A, B ) , a stabilizable pair, let 

A Tf 

LQ(A, ^) = {K > ^|k = K(A, Rj H ) for some R > D and some 
T T 

H so that (H , A) is a detectable pair} 

where K(A, B, R, H^) denotes the unique positive semidefinite solution to 
the algebraic Riccati equation: 

KA + a\ - KBR~ ^ b\ + HH^ = 0 

For R fixed, we denote the corresponding set as LQ(A, JB; R) . 

Definition 7 

SgCE) = {G^ e = -R“^b\, R > jO, K € LQ(A, R) } 

Proposition 3 - 

Given any stabilizable system Z(A, ^) , E ^3(11) implies (A + B^) 
is stable. 

Remark: The above proposition merely summarizes the well-known "standard" 

results of LQ-optimal feedback theory (refs. 1 and 4 ). However, the interpre- 
tation here of the LQ-optimal feedback class (83(1)) as a parametrization of a 
subclass of stabilizing feedback is interesting. 
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III. STRUCTURAL STABILITY CHARACTERIZATION OF LINEAR QUADRATIC (LQ) 

OPTIMAL FEEDBACK MAPS 


In this section, we show how the parametrlzatlon scheme developed pre- 
viously can be applied to obtain a characterization of the closed-loop struc- 
tural stability properties of systems under LQ-optlmal feedback. More pre- 
cisely,, we; establish an explicit parametrlzatlon of a general class of 
structural perturbations In. the control feedback gains as well as In the con- 
trol actuation matrix (^) that leave the closed-loop system stabilized. These 
new results, we believe,. are the natural generalizations of some earlier 
results of Anderson and Moore (ref. 2). 


We begin by first recalling from Lemma 2(111) that, for A stable, 

K € LP(A) always Implies K E LP(A - BLB^ K) ; however, for A unstable and 
K > ^ such that (A - BLB* ^) Is stable. It need not be true that 
Ke LP(A - BLB^ K) . The following example underscores this unfortunate state 
of affairs: 


Example: 


Then (A - BLB 


■ [: -’J 


BLB 


■[: 3 ' -[: 3 


Is stable, but 


K(A - BLB 


C T T P 3] 

K) + ( A - BIR K) K “ J ^ 0 


However, we have the following interesting observation. 


Lemma 4: 


K E LQ(A, B; R) =»• K E LP(A - M"^B^K) 


Proof: 


Immediate from the Rlccatl equation. QED 

In other words, the above unfortunate state of affairs cannot occur If 
K Is an LQ solution. 

We are now ready to state our first main result of the section. 

Theorem 3 (Infinite gain margin property) : 

a 

Let K E LQ(A, B; R) . Then {A - [B(^ + L);^"^ + B(N + M)B^]K} is stable 
for all L i R"^, M > 0, S = R(^) C R(L - R“^) 
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Proof; 


N = -n"^, R(N) C r(m) 
^ arbitrary 


We have K £ LP(A - so by Lemma 3, [A - + (V - W)K] 

is stable for all W S 0, V = such that R(V) C R(w) . 

Take W = B(L - + BMb' ^ and V = BSB^ + BN§^ and we are done. 


QED 


Remark: For ^ theorem 3 is a generalization of the "infinite gain 

margin” property of LQ-optimal feedback for single-input systems first noted 
by Anderson and Moore (ref. 1), who showed that the feedback gain vector 
_gT = -d/r)bTK can be multiplied by any scalar a > 1 without destroying 
stability; the proof they used involves classical Nyquist techniques. 

Theorem 3 not only generalizes this property to multiinput systems, but allows 
more complicated alterations of the feedback gain vectors. Moreover, it makes 
the proof of this property much more transparent. 

Remark; For B # theorem 3 allows for changes in the ^ matrix itself 
without destroying stability. One useful interpretation is the following. 

Suppose that the optimal feedback gain matrix has been computed for a 
nominal but that the actual value of B during system operation is 

changed to ^ ^ • Then the feedback term becomes 

So long as B^ = BqCN + M)R for some N = -N^, 

M > theorem 3 guarantees that the system will remain stable (e.g., 
a > 0). More complicated cases are allowed. 

Remark: Alternatively, the case _B ^ ^ can be interpreted as allowing for 

the possibility of adding extra controllers, and using these extra feedbacks 
to "fine-tune" the closed-loop behavior of the original system. (A more 
systematic exploitation of this idea will be dealt with in a future publica- 
tion; see also ref. 3). 

Theorem 3 has dealt with the case when the "negative" feedback gains, 
etc., are allowed to increase in magnitude; the converse situation, when the 
"negative" feedback gains are reduced in magnitude (or when additional "posi- 
tive" feedbacks are injected) is examined in the next proposition. 

Theorem 4 (gain reduction and robustness property): 

Riccati solution to the LQ problem (A, JB, R, Q) 
where R > ^ and (^^/^, A) detectable. Then 

(i) [A - ^(M + N)JB^R] is stable for all M > ^ such that 

M > -I R"1 N = -n'^ 
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(il) If A) Is actually observable, then 

[A - ^(M + + N) ] is stable where M, N are as above, and 

^ Is such that ^ < (1/2)^, R[(l/2)^ - ^) ] ^ R(^) and ^ is such 

that R[(l/2)^ - 3 R(&). 


Proof: 

(i) Let Zj. = I[(A - B] - Z(Ac, B ) . Then we have 

K G LP[Z(,; (1/2)R”^] from the Riccati equation; so, by theorem 2, 


[Ac + B(M - N)b’^K] is stable for all M < -| R"^ 


N = -N^ 

or [A - B(R“1 - M + N)b\] is stable. Let M - R“1 - M > <1/2)R-1, and the 
proof is complete. 

(ii) Let Ac = [A - ^(M + From the Riccati equation, we have 

KAc + + ^(2M - R“^)b'^K + ^ = 0 . 

Since A)-observable implies K > J0» K“^ exists, so we have 

K[Ac + + i)] + [Ac + R"^(^+i)]^K + M(2M- + (£- 2£) = 0 

Hence, subject to the condition (1/2)^ > R[(l/2)^ - ^ R(5. ^ N) it can 

be shown that 

- 2^) + ra(2M - R"^)B%, ^ + K"l(^ + N)] is observable 
Thus, by Lemma l(iii), [Ac + K"^(Q + N) ] is stable. 

QED 


Remark: Theorem 4(i) is a generalization of the known gain reduction toler-- 

once property of LQ-optimal feedback. This interpretation is most ^transparent 

in the special case when R"^ = diag (a^, a^) and M = diag (a^, ^) , 

N = ^. Then the original individual feedback loops are of the form 

Hi ” -a-jbj^*^K, i = 1, . . . , m. The theorem states that, in this special case, 

the system remains stable if the feedback gains are reduced to ^ 

so long as a^^ > (l/2)a^. More complicated cases are, of course, allowed. 

Remark: By interpreting the additional term + N) as a model perturba- 

tion term 6A of the open-loop matrix A, we can use theorem 4(li) to perform 
finite perturbation sensitivity analysis. The following simple example illus- 
trates the usefulness of this approach. 


Example: 


Let A 
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If we take 



Then we obtalm the algebraic Rlccatl solution as 



and the optimal feedback gain = -2[1 1], For any 



where 


Y € 

we are assured, by theorem 4(11), that 


3i 01^1 p.5 1 

< 

012 h 13 


[ 0.5 + 0j 0J2 + y“| 
hi-y -2 + ^ 2 ] 


+ abg* 


A + 6 A 

Is stable for all a > 1/2. Consider the following special cases: 
(a) Y = 0 J 2 * 

.T 


“0.5 2012' 

.0 “2 . 


+ abg* 


A + 6 A 

stable for all a > 1/2 and 0^2 that 


(1 - 0i2>^ < 1.5 or 1 - -j/l < 0,, < 1 + 


2 *"12 


yf 


(b) Y = 0J2 “ 


0.5 + 0j 0 " 

0 “2+02 


+ abg* 
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stable for all a > 1/2 and 3^, 32 so that 

( 1 ) 3 ^ < 0 . 5 ^ 3 ^ < 

(ii) (0.5 - 3i)(3 - 32 ) > 1 

Thus if 3^ = 0, the perturbed system is stable for all 32 < !• Other more 
general cases are, of course, allowed. 

The above example thus shows that the combined effect of feedback gain 
reduction and perturbation or uncertainty of the open-loop system parameters 
(poles and coupling terms) can be tolerated by a linear quadratic design with- 
out leading to closed-loop instability. Th±s vobus tries s prbperty of the 
LQ- feedback design deserves more attention. 


IV. CONCLUDING REMARKS 


Since further applications of the parametrization results established 
here to reliable stabilization synthesis and decentralized stabilization 
coordination will be made in a future publication, we reserve a fuller discus- 
sion of the implications of our approach until then. However, we would like 
to point out an important implication for pvaot'ioat design that is immediate: 
the ability to perform feedback "loop-shaping" analysis. . 

In any realistic synthesis problem (keeping a system stabilized, local- 
izing particular disturbances, etc.), there is usually a large number of fea- 
sible solutions. While the use of cost-criterion optimization (e.g., LQ) in 
theory allows the designer to pick exactly one such solution, in practice, the 
difficulties of judging or fully incorporating the relevant cost considera- 
tions and their tradeoffs as well as the often gross model uncertainties and 
physical variabilities of the system and the controllers necessitate further 
sensitivity analysis or trial- and-error "hedging" about the nominal solution. 
It is therefore very important in the computer-aided desi^ context that the 
"feasible solution space" structure be known in some detail to facilitate and 
guide the conduct of iterative search. In this regard, a major merit of a 
"classical" design technique like root locus is that it provides an explicit 
functional dependence of the closed-loop system structures (distribution of 
poles and zeros) on the control structure (feedback gain). However, such 
classical approaches become totally intractable when there is a multiple num- 
ber of controllers, while "modern," "state-space" linear feedback design tech- 
niques like "pole-placement" algorithms and "dyadic-feedback" design suffer 
the serious drawback of providing little structural information about the 
solutions they generate. Moreover, such techniques are guided more by mathe- 
matical convenience than by physical interpretation. From this perspective, 
the parametrization results established earlier show promise in providing the 
basis for a new iterative design algorithm that will overcome the last- . 
mentioned drawbacks. 

o 

Several years ago, Rosenbrock (ref. 6) suggested .a frequency- domain 
multiloop feedback design technique ("inverse Nyquist array" method) which he 
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motivated also as an attempt to overcome some of the abovementloned drawbacks 
His approach is in contrast with ours, which Is a "time-domain" approach. It 
will be Interesting to investigate the connection. If any, between the two 
approaches. 
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ANALYSIS OF PILOT- AIRCRAFT PERFORMANCE AND RELIABILITY 
VIA THE APPLICATION OF MAN-MACHINE MODELS 
David L. Kleinman 
University of Connecticut 

SUMMARY 


Analytic modeling of human performance has progressed to the point where 
it can be used with some confidence in the design and performance analysis of 
man-machine systems. This presentation first reviews the present approach to 
this analytic modeling, which utilizes human response theory together with 
modern control theory. Then an example of its use in helping design a flight 
director for the hover control of a VTOL vehicle is described. Finally, how 
the method could be used for analysis of human response to system failure is 
sketched. 
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Objectives 


• Develop and apply analysis procedures for manually controlled systems 

• Understand human as an Information processor » controller and decision 
maker 

Approach 

% Combine human response theory with modem control and estimation theory 
Assumption 

• Experienced human behaves optimally subject to his Inherent limitations 
Applications to Manned Vehicle Systems 

• Basic manual control tasks 

• VTOL precision hovering 

• Failure detection and control 

• Carrier and land-based approach 

• Anti-aircraft gunnery 

• STOL flare and landing 

Figure 1.- Modem control approach to human modeling. 


input 

disturbances 



noise noise 


Figure 2.- Man as a generalized feedback control element. 
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X(t) = Desired state trajectory 

x(t) = Deviation of system response from nominal 

Linearization about X(t) 

x(t) = Ax(t) + Bu(t) + Ew(t) + Fz(t) 

■■ ■ *— "..V . 

disturbances 

w(t) Gaussian» zero mean white noise 
E{w(t)w(r)} = W(t)5(t - t) 
z(t) = Non-random, bias inputs 
li(t) = Ajj h(t) + Fh z(t) 

Y(t) = Available Information concerning x(t) 

contains position and velocity Information 
Y(t) = Cx(t) + Du(t) 

Figure 3.- Vehlcle/dlsplay dynamics. 

Lump sensory, central processing, perceptual 
limitations Into perceptual equivalents 

% Lumped human time-delay, t 

• Small signal thresholds, F(*) 



• "Observation" and central processing noise, Vy 
Yp(t) = "perceived" signal 

Yp(t) = F[Y(t - t)] + Vy(t - t) 

Figure 4.- Human perceptual limitations. 
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• Most Important element In human randomness i 

• Related to subject training level 

• For foveal viewing of a single display 

Vyi(t) = ^Pyi E{Y|(t)} 

= nolse/slgnal ratio 

• Scanning and attention allocation 

= fractional attention to display k 


NI 



Human picks to optimize information 


Figure 5.- Equivalent observation noise. 


• "Neuro-motor" dynamics; h(u.y ■= 
o(u) + Tjj u + u <* 

Uq = commanded control 
« = actual control 


• Implicit modeling via limits on control rate 

• "Motor" noise, 

White, Gaussian noise 

Second component of human randomness model 


+ M = 


- Cui ■ 


Figure 6.- Human control limitations. 
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L J 

Human operator model 


Figure 7.- Optimal control model of human 
response • 


I 


Tjju + u » -Lx(t|t) + V^Ct) 

Tjj “ "neuromotor" lag from u weighting 
L = optimal feedback gains 

x(t|t) = best estimate of x(t) from {Yp(o) , o < t} 

Generation of x(t|t) constitutes human's Information 
processing behavior 

• Kalman filter 

Generates best estimate pf x(t - T|t) 

Compensates for. VyCt) 

• Predictor 

Generates x(t|t) from x(t - T|t) • 

Compensates for delay 

• Separable feedback control behavior 

Figure 8.- Feedback control solution. 
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• Time delay, t 


0.15-0.25 sec 


• Observation noise ratios, p, • ss 0.01 (-20 ± 3 dB) 

yT' 

• Thresholds, ai 0.05* visual arc 

(position) 
0.05*/sec (rate 

• Motor noise ratios, — 0.003 (-25 dB) 

• "Neuromuscular” lag, 0.05-0.12 sec 

Assumption ; Htmian parameters do not depend on 
vehicle d}mamlcs or control cost 
functional 

Human parameters do depend on external 
Influences — light, heat, stress, vibra- 
tion, display resolution, etc. 

• Applications 

Figure 9.- Typical human response parameter values. 



• Typical Y^, = fc, k/s* k/s^ 

• T(s) * noise shaping filter 

= a?l + w z/j “ «2 

a ?2 = + « z /2 “ .^2 “ : 

• Human's task: minimize mean squared error 

J(u) = E[z/f + g u^] 

• Steady-state, stationary behavior 


m(s) = ^i(s)i/i(s) + h 2 (.s)y 2 is) + noise 
= 1^1 (s) + 8 h 2 (. 8 )]y^i 8 ) + noise 


measurable 


Figure 10.- Simple SISO tracking tasks. 







147 



Frequency, rad /sec 


Figure 11.- Measured and predicted performable measures 
k/s^ dynamics (Avg of 3 subjects). 


• . Refinement of existing model 

Analyze model-data mismatches 
Effect of a priori Information 
Attentlonal allocation/scanning 

• Human decision making 

Decisions based on x 
Fault detection 
System monitoring 

• Human adaptivity and learning 

Explore relation to observation noise 
Training effects 

• Environmental effects 

Relate to changes In human parameters 
Stress, age, alcohol, vibration, ^-forces 

• Multi’-human control tasks 

• Human modeling In socio-economic systems 

Figure 12.- Further efforts in man-modeling. 



• Objective: allocate tasks between pilot and automatic system 

• Approach: 

1) Select pilot /automatic configurations 

2) Determine information requirements for control and monitoring 

3) Perform comparative evaluatJLon 

Figure 13.- VTOL control/display design. 



• Monitoring workload metric 

f = 22 “ monitoring workload 

® i 

, fMi ~ fractional monitoring attention 

on ith display 


Figure 14.- Pilot monitoring with fixed total workload. 
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• Equal attention 


\ “ fM/"y 

Equivalent to residual monitoring 

• Peak excursion 

tlyil ^ 

• Generalized quadratic index 



• Instrument failure detection 

• Relative estimation errors 

• Performance assessment 
• Miscellaneous 


Figure 15.- Monitoring model concepts. 


• Control performance predictions 


Jc - 2 


max 


• Control workload metric 

= control 

i ^ 


workload 


f^^ = fractional control attention on ith display element 

• Attention allocation 
min 


Figure 16.- Optimal control pilot model. 
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X = Ax + BUj. + Ew , y = Cx , uc = -Lx 

Jc = + trELgZL^] Lg = diag(q^l/2)Le^'^ 

“ V t trCCeEC^] = diag^^^Ce^'^ 

0 = ZA' + AZ + EWE’ - ZC’V^^CZ 
Control: ; 0^ = gCf^. * * •> 

Monitoring: V, = p a.2/(f^ H- fjjj )N. ; a,, = constant 

111 i i ^ ^ i 

Figure 17. ~ Expressions for jcost measures J^, Jj^. 


• Closed-form expressions for gradients 



G = ZC^V"^ = Kalman filter gain; A = A - GC 

• Nonlinear threshold effects included as RIDF 

• Gradient projections 

Zfj^ = constant; = f^ for positions/rates 

• Basic gradient optimization for f^ 

Figure 18.- Algorithm for optimizing f^. 
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* Longitudinal axis, pitch hold, * 0.3 

• Predictions (A) without FD, (B) with FD 


Instrument 

0 

^1 

fci/fc 

w/o FD 

FD 

w/o FD 

FD 

Map 

25.5 ft 

16.6 ft 

0.35 

0.03 

Altimeter 

4i7 ft 

4.7 ft 

.17 

.1 

Pitch 

1.0" 

.7" 

.03 

.03 

IVSI 

lJ4 fps 

1.2 fps 

.45 

.13 

FD^ 

— 

.13 In. 

— 

.30 

FDh 

— 

.14 In. 

— 

.40 

«e 

.13 in. 

.10 in. 



«c 

.25 in. 

1 

.22 In. 




Figure 19.- Helicopter hover task. 



Figure 20.- Predicted performance CH-46 hover flight director used. 
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• Performance measures and workload metrics defined for control and/or 
monitoring 

• Procedures developed for attention allocation - avoids previously arbitrary 
choice of f^ 

• Continuing efforts 

• Further validation 

• Determine information requirements 

• Tradeoff studies 


Figure 23 .- Sxunmary. 


Context 

Detection and identification of unexpected system failures 
Objective 

Model the human’s decision process and adaptive control behavior 
Combine 

• Human operator modeling 

• Decision and control theory 

• Human response results 

• Planned experimentation 

Figure 24.- Models for human decision-making and control. 
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Human adoptive process 



Figure 25.- Human adaptive process. 

t < tf : x(t) = A(jx(t) + BQu(t) + w(t) 

y(t) = Cox(t) 

t > tf : x(t) = AjjX(t) + Bj^u(t) + w(t) 

y(t) = C^^x(t) 

j i“0j i ...} N} 

Humans task ; Continuously determine true 
Models ; 1. Sequential failure detection and Identification 

2. Simultaneous failure detection and Identification 




Figure 26.- The adaptive control problem 







• Internal dynamic model in Kalman filter 

p(t) = Ap(t) + Bu(t - t) + Gfyp(t) - Cp(t)] 
p(t) = x(t - t) 

• State estimate x(t) 

x(t) = best internal estimate of vehicle state 

K. ^v(t) = error covariance 

(x*~x) 

• Residual or Innovations r(t) 

= yp(t) - Cx(t - t) 

Nominally r(t) = zero mean, white noise 
Figure 27.- Features for adaptive modeling and decision making. 


•If = So then r(t|So) is zero-mean, white, with covariance Vy 

• Testing r(t) over t - T < t < t 

1 . Mean value 

2. Whiteness 

3. Covariance 

• Determine model parameters via experiment 

• Limited model 

Figure 28.- Failure detection by innovations testing. 
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% Model structure for control and decision 


% Experimental planning, model validation 
% Implications for automation 

1. Human monitoring 

2. Automatic status Information 

3. Allocation of decision making 

Figure 29.- Conclusions. 
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RECENT PROGRESS IN MULTIVARIABLE CONTROL 


W. M. Wonham 

University of Toronto, Canada 


SUMMARY 


The last five years have witnessed a remarkable development in linear 
multivariable control, from both the theoretical and applied viewpoints. This 
paper summarizes the major new results in structural synthesis and design 
which are now available in textbook form, and suggests a number of promising 
directions for future research. 


1. INTRODUCTION 


The last five years have witnessed a remarkable development in linear 
multivariable control, from both the theoretical and applied viewpoints. 
Deferring for a moment the applied aspect, we note first that significant 
advances have taken place in our theoretical understanding of system struc- 
ture. Under this heading we group two major topic areas: realization theory 

and system synthesis. 

Realization theory is concerned with the axiomatic definition of "system," 
and with relating "internal" system descriptions (i.e., state descriptions) 
with "external" (i.e., input-output) descriptions. For this, a unified mathe- 
matical framework was supplied by Kalman et at. (ref. 1), exploiting the con- 
nections (well known to algebraists) between linear algebra and the theory of 
modules over a polynomial ring. The general theme has since been pursued in 
the context of delay systems (ref. 2), bilinear systems (ref. 3), and what 
might be called "categorical" systems (ref. 4). 

While realization theory supplies the infrastructure, it is system syn- 
thesis and design that is perhaps of most direct relevance to engineers, and 
the remainder of this paper is devoted to sketching the situation here, 
together with the implications for practical applications. Before getting 
down to specifics, it may be useful to distinguish between "synthesis" and 
"design," at least in the context of control systems (cf. ref. 5, ch. 2). 

I would label synthesis as the process by which one establishes the 
qualitative structural possibilities: that is, one determines whether and how 

one can achieve such desirable'* system properties as noninteraction (among 
functionally distinct subsystems and their associated controllers), loop 
stability, regulation of specified outputs with respect to disturbances, 
tracking by specified outputs to reference signals in some preassigned class. 
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and, finally, structural stability (i.e., preservation of the foregoing prop-, 
erties within a range of parameter variations). 

On the other hand, design would refer to the numerical massaging 
(ideally, optimization) of system parameters, within the structural frame- 
work established by synthesis, to meet quantitative design specifications 
related (in this context) to transient response, stability margin, saturation 
levels, and so on. While the distinction is obviously not clear cut, we have 
at least two working definitions. 

The point of making them is to stress that a control theory, qua theoret- 
ical construct, deals largely with synthesis, and thus may be expected to 
operate on a level of relative generality and abstraction. Its function is 
precisely to clarify the structural possibilities. Design may then bring in 
a host of special techniques, many of them heuristic, to ensure that a sped- 
fic gadget works in practice. It is hoped that these simple-minded remarks 
may forestall useless arguments about the ”gap." 

We shall now review some recent theories of linear multivariable system , 
synthesis, turning finally to their potential impact on design. 


2. APPROACHES TO SYNTHESIS 


A theory of synthesis will start from an assumed system model or class 
of models together with an assumed class of admissible controls, and supply 
constructive answers to a series of precisely posed questions of the following 
kind: Does an admissible control exist such that the resulting (synthesized) 

system displays such-and-such desirable characteristics? To qualify as a 
theory, and not just a library of algorithms, our constructive procedures . 
should somehow exhibit the answers in terms of a reasonably small number of 
basic system concepts. While these must, of course, be made theoretically 
precise, they must also embody enough intuitive content to render the process 
of discovering new answers, by new procedures, easy and natural for the prac- 
titioner. Finally, the variety of questions amenable to attack must be wide 
enough to embrace a significant range of potential applications, and our con- 
structive procedures must translate readily into computational algorithms: 
these will provide the framework for design, as noted earlier. 

I think it is fair to claim that two complementary theories exist today 
which go some way to meeting these criteria. These are the geometric state- 
space approach (ref. 6 and references cited therein), and the polynomial 
matrix-frequency domain approach (ref. 5, 7, 8, and cited references). 

I should emphasize that in neither of these theories is the notion of 
optimization at all central, and I omit discussion of the familiar **linear- 
quadratic" optimization problem, on which an extensive textbook literature 
already exists. It suffices to say that quadratic optimization is merely one 
method of achieving the rather limited goal of stabilization via state 
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feedback » and this theory, useful as it may be, contributes little to the 
primary objectives of structural synthesis. 


2.1 Geometric State-Space Theory 

Here we take for granted a (discrete or continuous-time) state model, of 
the familiar type 

X = Ax + Bu + Er + . . . 
y = Cx + Cr + . . . 
z = Dx + Du + . . . 

As usual, x and u denote the state and control vectors, r - r(t) is a 
disturbance vector, y is the vector of measured variables, and z the vector 
of variables to be regulated (e.g., tracking errors, deviations from set-point 
values). We restrict attention to the deterministic theory, although the 
stochastic counterpart is fairly straightforward. 

In purely linear theory, the admissible controls are of the type 

u = Fx + Gr + . . • 

Incorporating state feedback and (possibly) disturbance feedforward. There 
will be a constraint restricting the control to process only the measured 
variable y; this translates algebraically as 

Ker(F,G) D Ker(C,C) . 

One can bring in observers or other dynamic compensators by various standard 
tricks • 

What are the basic system concepts? Everything starts from the funda- 
mental ideas of controllability and observability or, more accurately, the 
geometric objects “controllable subspace" and "unobservable subspace" of the 
state space x. Two other related families of subspace that play a basic 
role are the " (A, B)- invariant subspaces" and the " (A, B) -controllability sub- 
spaces." 

To give a brief indication of their meaning, suppose we have simply 

X = Ax + Bu , z = Dx . 

Then we say that the subspace V c Ker D is (A, B) -invariant if, whenever 
the initial state x(0) G V, there exists a control u(t) (t ^ 0) such that 
the ensuing state trajectory x(t) = x(t;x(0),u) also satisfies 
x(t) e V (t > 0) ; then Dx(t) = 0. It turns out that there is always a 
largest (A, B) -invariant subspace, say V*, contained in Ker D: roughly, this 
is the largest set of states which can be made unobservable (by suitable 


159 



control) at the output z. Finally, the subspace R C X is an (A,B)- 
aontrotlabiUty subspaae if every x g R can be reached from the origin of 
X by a; controlled state trajectory wholly contained in R: thus, if 

R C Ker D, you can sneak from zero to any x G R without being seen from z. 
There is always a largest R, say R*, in Ker D, and we always have R* C V*. 

Using just these natural and elementary ideas, it has been possible to 
build up an elaborate set of procedures for structural synthesis. What 
problems have been solved? 

Historically (1970) , the first was the longstanding problem of noninter- 
acting control or decoupling. In the simplest version, one is given the 
system 


X = Ax + Bu 


with output vectors 


z,=D.x, i=l, ...,k 

X X ’ 

One asks for a state feedback control with new external inputs v±, 

k 

u = Fx + G.v, , 

so that each Vi controls the output z± alone without affecting the Zj 
for j i. Under very reasonable conditions on the number of available 
controls, one can show that this is almost always possible (at least by 
dynamic compensation, omitted here); furthermore, one can achieve more-or-less 
arbitrary pole locations for the closed-loop system. The condition for 
solvability is just that 

R^* + Ker D^=x, i=l,...,k 

where R^* is the largest controllability subspace contained in 

n Ker D. 

The numerical checking of this condition is quite simple. At the University 
of Toronto, we have developed APL routines that have been run successfully on 
decoupling problems of at least moderate dynamic order (15-20). 

The second major structural problem to be treated completely and suc- 
cessfully by these methods was the general multivariable servoregulator 
problem (1974) . One is given 

X = Ax + Bu , y = Cx , z = Dx 

Here the dynamics represent the plant together with a dynamic model of the 
disturbance (and/or reference) signals that must be rejected (and/or tracked). 
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One asks for a control u = Fy so that z is regulated (z(t) 0 from any 

initial state) and so that the loop is stable. While these two requirements 
are not necessarily compatible, it turns out that they almost always are, 
provided one has at least as many scalar controls as scalar variables to be 
regulated. The precise solvability conditions are a little too involved to 
state here, but they amount to quite transparent relations among suitably 
defined V*, R*. Again, it is straightforward to develop a synthesis proce- 
dure that translates, for example, to APL (ref. 9). 

Limitations of space prohibit going into further detail: may I just men- 

tion that the geometric theory has also shed light on decentralized stabiliza- 
tion (ref. 10) and to various synthesis problems involving system "zeros” 

(ref. 11). In summary, the "geometric approach" has already justified itself 
in terms of results and shows ample promise for future development. 


2.2 Polynomial Matrix Formalism 
On taking Laplace transforms of the equations 

X = Ax + Bu 
z = Dx + Du 

one is led to the polynomial system matrix*. 



More generally, polynomial system matrices of form 

Tt(s) U(s)1 

P(s) = , det T(s) t 0 

L-V(s) W(s)J 

arise when higher order differential operators appear in the initial dynamic 
description. The more familiar transfer matrix is now defined as 

G(s) * V(s)T“l(s)U(s) + W(s) 

The rich structure theory of P(s) and G(s) has been explored notably 
by. Rosenbrock (refs. 5 and 7) and Wolovich (ref. 8). As these authors adopt 
somewhat different viewpoints, we shall review their approaches separately. 

Of course, in the space available we cannot do them justice. 

In terms of P(s), Rosenbrock supplies and justifies the appropriate 
definition of system dynamic order (=deg [det T(s)]), together with that of 
system poles and zeros, and relates these concepts to the controllability- 
observability structure of a state variable realization. By the introduction 
of "strict system equivalence" (based on coordinate transformations involving 
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no change in dynamic order) , it is shown how P(s) can be reduced to various 
useful standard forms; and then how the invariant structure of P(s) relates 
to that of G(s) (e.g., the invariant polynomials of P coincide with the ■ 
Smlth-McMillain numerator polynomials of G). (In this review, some accuracy 
is sacrificed to brevity.) The system matrix is further exploited to discuss 
open-loop zero assignment (by output selection) and closed-loop pole asslgn-r 
ment (by state feedback). 

The formal theory culminates with an Investigation of the key idea of 
diagonal dominance. A rational matrix Q(s) = [qi 4 (s)] is diagonally {tow) 
dominant on a contour C in the complex s-plane if 

kii(s)| - 22 kii(s)| > 0 

for all i, and s G C. Rosenbrock now shows that If the forward-loop trans- 
fer matrix is diagonally dominant (with C a standard Nyquist path) , then 
stability of the closed-loop S 3 ^stem can be studied by application of the 
Nyquist (or inverse Nyquist) criterion to the diagonal elements alone. 
Coupling effects are accounted for by broadening the usual Inverse Nyquist 
loci into “bands,” obtained by ingenious exploitation of various inequalities 
due to Gershgorin and Ostrowski. 


While something will be said below about the design methods that ensue, 
this summarizes the "synthetic” content of Rosenbrock* s approach (refs. 5 
and 7 ) . 

What are the basic system concepts? — again, controllability and observ- 
ability; next, the polynomial system matrix and its equivalent forms; and, 
finally, diagonal dominance or quasi-noninteraction. 

What synthesis problems have been solved? Actually Rosenbrock does not 
pose synthesis problems in a highly formal way, but proceeds more in the 
spirit of "design” to illustrate how the foregoing concepts can be exploited 
to satisfy various closed- loop requirements. Heuristics are proposed to 
achieve open-loop diagonal dominance through simple types of plant precompen- 
sation. Here interactive computation is essential, with graphical display of 
inverse Nyquist arrays. The result is to reduce the design problem to 
decoupled "classical'* subproblems for independent single- input/single-output 
loops. It is reported (convincingly) that criteria of stability margin, 
bandwidth, structural integrity, and so forth can be effectively brought into 
play. 


Turning now to Wolovich's approach (ref. 8), we find a creditable attempt 
to unify the state and frequency viewpoints. The starting point is a "struc- 
ture theorem” that represents a transfer matrix G(s) in the form 

G(s) = N(s)D“^ (s) 
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where N(s) and D(s) are polynomial matrices that satisfy a certain coprime- 
ness relation. Under the action of linear state-variable feedback, it turns 
out that the "numerator** matrix N(s) remains unaltered, while the "denomina- 
tor** matrix D(s) is almost freely assignable. In this way, the effect of a 
time-domain compensation scheme can be displayed in the frequency domain. 

A furtlfer result shows that any control scheme that can be implemented by 
feedforward compensation can also be implemented via an equivalent feedforward/ 
feedback scheme. 

With these as the basic concepts, Wolovich turns to a series of synthesis 
problems, formally posed in terms of transfer matrices. The major problems 
are noninteracting control (decoupling) and exact model matching. The decou- 
pling problems are resolved by constructive procedures; for model matching, 
heuristics are proposed. It is worth remarking that, while Wolovich’ s 
expressed intent is to unify the state and frequency approaches, his transfer 
matrices are treated as purely algebraic objects: the usual frequency domain . 
ideas (e.g., bandwidth, gain, and phase margins) never appear at all. In 
effect, his procedures represent disguised linear algebra, with the poly- 
nomials serving mainly as bookkeeping devices. Of course, this is not to say 
that the procedures, once mastered, may not be effective; presumably, they are 
most convenient when system data are supplied initially in transfer matrix 
form. 


3. IMPACT ON DESIGN 


My remarks here must be brief and tentative, as the methods just 
described are quite recent. One thing is clear: as the methods have been 
evolved for application to large, complex systems, design will rely on inter- 
active computing, presumably with graphical display (of Nyquist arrays, pole- 
zero patterns, time and frequency responses, etc.). For this, development of 
efficient numerical algorithms and their organization via a high-level proce- 
dural language is the kind of major advance to be expected in the near-term 
future. Considerable progress has already been achieved by, for instance, 
Rosenbrock at the University of Manchester (U.K.), Astrom at the Lund Insti- 
tute of Technology (Sweden) , and Mansour at the ETH (Zurich) . These three 
groups are sustained in part by substantial interaction with industry. Men- 
tion should also be made of the work in computer-aided design at Cornell, as 
reported by Merriam (ref. 12). 

4. CONCLUSION 


Large-system multivariable synthesis has arrived. The outlook for 
attacking the further problems of hierarchical and decentralized control is 
good. The effective exploitation of what is known even now will, neverthe- 
less, demand patience, and commitment of resources. 
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A SUMMARY OF NONLINEAR FILTERING 


^ Ian B. Rhodes 

Washington University, Mo, 


I. INTRODUCTION 


In this summary paper, we attempt to highlight the main results of modem 
nonlinear filtering theory and discuss some of the approximations commonly 
made to yield suboptimal filter designs (such as extended Kalman filters) that 
avoid the infinite-dimensional on-line computations required of the optimal 
nonlinear filter. No attempt is made to provide a survey of the field of non- 
linear filtering or even to begin to approach the breadth and depth of cover- 
age or the exhaustiveness of referencing a survey implies. Nor does this 
paper provide the self-contained development conventionally thought of as 
being implicit in a tutorial introduction. We content ourselves with outlin- 
ing the main approaches, stating and discussing the main results, and refering 
to a representative sampling of papers or books in which derivations, proofs, 
details, and extensions can be found. These considerations plus a dedire to 
be brief leave great latitude in the selection and relative emphasis of topics 
and, perhaps even more than most papers of a summary or survey type, this 
paper reflects a personal view of the field of modern nonlinear filtering. 

This paper is organized as follows: section II is devoted to the formu- 

lation of the prototype nonlinear filtering problem with which the rest of the 
paper is concerned, along with a discussion of some of the modifications, 
extensions, and alternative models found in the literature. In section III, 
we outline the main approaches used to characterize or find the optimum non- 
linear filter; in section IV, we recall the enormous simplifications that 
result when the system is linear. Sections V and VI are concerned with 
approximations to the optimal nonlinear filter: the former with relinearized 

estimators and extended Kalman filters, the latter with approximations to the 
conditional density. Finally, section VII summarizes an approach to designing 
and evaluating the performance of suboptimal filters in terms of upper and 
lower bounds. 


II. FORMULATION OF NONLINEAR FILTERING PROBLEM 


For the nonlinear filtering problems considered here, attention will be 
concentrated on the smooth, finite-dimensional, continuous-time dynamic 
system: 

dx^ = f(x^,t)dt + D(t)dv^ (la) 

dz^ = g(x^,t)dt + G(t)dw^. (lb) 
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where x^. is an n- dimensional random vector, is r-dimenslonal, and 

wt are m- dimensional, and v and w are independent normalized Wiener pror 
cesses. The initial state Xq is independent of v and w and is Gaussian 

with mean Xq and covariance Eq. Functions f and g are assumed to satisfy 

suitable technical properties; G(t) is nonsingular. Everything takes place 
on the fixed finite interval [0,T]. Less rigorously, we can also write 

= f(x^,,t) + D(t)v^ (la*) 

y^ = = g(x^.,t) + G(t)w^ (lb*) 

where the formal derivatives v and w of the Wiener processes v and w are 

(independent) Gaussian white noises. The system equation (1*) can be obtained 
formally be dividing both sides of equation (1) by dt and thinking of y = z 
as the observed processes. Similar formal manipulations can be used to con- 
vert, for example, the filters written later in the *’dif ferential” notation 
used in equation (1) to the ‘'derivative** form used in equation (1*). 

We will also restrict attention to minimum-mean-square-error filtering. 
Letting be the past of the observation process z,^ that is, 

Zt ^ 0 < s < t> (2) 

we are thus interested in finding the conditional mean 

Xj. = E[x^ I Z^] (3) 

of xt given Z^, and the corresponding conditional covariance 

Et = cov[xt. I Zj.] = E[(X(. - x^Mxt - x^)' I Z^] (4) 

Throughout the paper, prime denotes transposition. The conditional covariance 
provides, of course, a measure of accuracy or confidence. 

The filtering problem defined by equations (1) to (4) has established 
motivation as a model that arises in a wide variety of control and communica- 
tion problems, and most of the literature in the systems area is concerned 
with this model or a modification of it. Some of the many possible modifica- 
tions and extensions include the following. 

(a) From a control viewpoint, the most important extension is to replace 
equation (la) by 

dx^ = f(x^,u^,t)dt + D(t)dv^ (la**) 

where the (r- dimensional) control is assumed to be derived causally from 

the output process z. Most of the results given later have natural counter- 
parts for this model. 

^More precisely, Z^ is the a-algebra generated on the underlying proba-* 
bility space by the observation process up to time t. 
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(b) Smoothing and prediction problems, where interest centers on finding 
E[xt I Zg]; prediction problems have s < t, smoothing ones have s > t. More 
generally, one can seek E[h(x|-) | Zg] for some suitable function h, and we 
can have s > t, s < t, or s = t. 

(c) The finite-dimensional continuous-time system (1) can be replaced by 
alternative system models; for example, by a finite-dimensional discrete-time 
system 


^+1 “ 

(5a) 

\ + G(k)Wj^ 

(5b) 


or by an infinite-dimensional system such as a delay-differential equation or 
a partial differential equation (refs. 1-6). Consideration of the discrete- 
time finite-dimensional model (5) avoids a number of technical mathematical 
questions associated with the continuous- time system (1); conversely, techni- 
cal problems are accentuated for infinite-dimensional models, even when they 
are specialized to being linear. 

(d) Generalization of noise process models: extensions are available 

for correlated noise processes v and w (v and w) , "colored" (rather than 
"white") V and w, and state-dependent noises where the distribution matrices 
D(t) and G(t) in equation (1) depend also on the system state. When f and g 
are linear, a "wlde-sense" approach can be added: here v and w are taken to 
be uncorrelated- increment processes (v and w wide-sense white) not necessar- 
ily Gaussian, and one seeks the tineccp minimum-mean-square-error estimator. 
Problems in which G(t) is singular lead to singular estimation problems^ 
only when f and g are linear has some headway been made (refs- 1, 5, 

and 7-11). 

(e) Different criteria: the minimum-mean-square-error optimality cri- 

terion can be replaced by other choices, such as maximum a postev'Lov'L (MAP) 
estimation, etc. Other modifications include formulating the problem as one 
of statistical least-squares estimation; invariant imbedding or the calculus 
of variations are typical solution techniques (e.g., refs. 12 and 13). 

(f) Alternative observation models arise naturally in some applications. 
For example, for certain problems in nuclear medicine and quantum-limited 
optical communication, the observations take the form of a point (counting) 
process usually modelled as a Poisson process whose intensity (rate) is itself 
a random process influenced by the system state x^ (e.g., refs. 14 and 15). 
More recently, martingale theory has been used to provide a unified treatment 
that includes counting process models and the model (1) as special cases 
within a more general setting (e.g., refs. 16 and 17). 

(g) Special structures: For some classes of systems, a vector space 

setting is less natural than a formulation in some other mathematical frame- 
work whose structure is better matched to the characteristics of the class of 
systems at hand. An example is the formulation of some problems on manifolds 
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and bringing to bear the techniques of differential geometry In attacking them. 
Included here are estimation problems Involving bilinear systems (refs. 18-20). 

(h) Combination problems: Estimation problems frequently arise In com- 

bination with additional requirements such as Identification or hypothesis- 
testing (the latter Includes detection and some formulations of reliability 
problems). Superimposed on top of these can also be a control task. 


III. APPROACHES TO SOLVING NONLINEAR FILTERING PROBLEM 


There are essentially two main analytical approaches to finding or char- 
acterizing the conditional mean and conditional covariance. The first Is to 
find or characterize the conditional density Pt(*t I Zj.) of Xj. given Z^, 
or the conditional distribution Pt(^t I ^t)* conditional mean and covar- 

iance then follow as 


J* xp^(x I Z(.)dx = r X dy^(x | Z^ 
R“ R" 

J (x - x^^)(x - x^^)' p^^(x 1 Z )dx 
V c t t 


( 6 ) 


= J* (x - Xj^)(x - x^^)' dy^(x 1 Z^) 

SP 


(7) 


There are. In turn, two main ways In which Pt(*t I ^0 characterized. 


(A) pj.(xt. |- Zj:) ' satisfies the partial differential equation 
d^pj.(x I Z^) = L*[p^(x I Z^)]dt + p^(xl Z^) [g(x,t) - g^]'W~l(t) [dzt - gt<it] (8) 


where 

W(t) 


A 


d- on the left side denotes differential with respect to time, 
G(t)G*(t), 

gt = E[g(x,t) I Z,^] = J g(x,t)p (x I Z )dx 


(9) 


and L* Is the forward Kolmogorov partial differential operator associated 
with the system (1) : 


L*[‘] 


-E 

•r si 


3[f^(x,t)* ] 


3x 


1 



n n 

EE 


1=1 j.=i 


3^ 

Sx^SXj 


[D(t)D’(t)-]^j 


(10) 


The stochastic partial differential equation (8) has been called Kushner's 
equation (refs. 21 and 22). It Is also referred to as a modified Fokker- 
Planck equation or a conditional Kolmogorov equation. 
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The practical difficulties in attempting to find PtC^t I from equa- 

tion (8) are the obvious computational problems of solving a partial- 
differential equation on line, Levieux (ref, 23) reported recently on his 
efforts in this direction; indications are that the approach is limited with 
current computational technology to a state x^ of dimension at most two or 
three. 

(B) A second way to characterize P^C^^ I Z^.) is via the so-called Bucy 
representation theorem (ref. 1) that relates the conditional density of xt 
given to the prior density p^(x^) of x^.: 

E{exp <()^ I x^ = x} 

I E{.xp I Zt> 

where (log likelihood ratio) ±s 

= JT g'(Xg,s)dZg - i 


The representation (11) can be viewed as a form of the Bayes rule suited to 
the problem under consideration here. Its validity rests on certain condi- 
tional independence assumptions satisfied for the prototype problem formulated 
in section II, but are violated if, for example, v and w are correlated or if 
equation (la") is substituted for (la) . Note that equation (11) is not as it 
stands an explicit formula from which p^C^ 1 Z^,) can be calculated but 
rather it is a relation that P|-(x | Z^) must satisfy since the coefficient 
of Pt(x) on the right side depends implicitly on p^(x [ Z^) . Note also 
that the numerator of this coefficient involves a conditional expectation in 
which attention must be confined to state trajectories that pass through point 
(x,t). We return in section VI to the use of representation (11) and its 
variations in constructing approximations to p^(x ] Z^.) . 


As an alternative to first finding the conditional density and using 
equations (6) and (7) to compute the conditional mean and covariance, one can 
instead write stochastic differential equations for x^. and 


dx^ = f^ dt + [x^g^(x^7^ - x^g^]\r^(t)[dz^ - g^ dt] 


(13) 


dZ^ = f(x^,t)[x^ - x^]Mt + [x^ - x^]f ' (x^,t)dt - dx dx’ + D(t)D’(t)dt 


+ [x^ - x^][x^ - X^.] * [g(x^.,t) - g^] 'w“^ (t) [dz^ - g^ dt] 


(14) 


where = E[f(xj.,t) | Z^.], etc. These stochastic differential equations can 
be derived in a number of ways, such as by combining equations (6) and (8) and 
integrating by parts, by integrating equation (11) by parts, by the innova- 
tions approach (ref. 7), or by measure transformation approaches (refs. 24-26). 
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However, the integratd^n_of_^quation (13) requires the simultaneous compute^ 
tion of f|-, g^, and x^g^(x^.,t; and, although stochastic differential equa-y 
tions for each of these can also be written down, they, in turn, require ^ 
simultaneous calculation of even more terms such as 

g^^ = E[g(x^,t)g’ (x^,t) I Z^], the stochastic differential equations for 
which require even more terms, and so ad infinitum. Similar remarks apply to 

Thus, except in certain special cases where this mushrooming requirement 
for additional terms terminates at a finite number, we are left with the need 
to solve an infinite system of stochastic differential equations. 

Thus, generally all three approaches to computing the conditional mean 
and covariance require an infinite-dimensional on-line calculation, whether 
that is the determination of the conditional density from the partial differ- 
ential equation (8) or via representation (10) , or the solution of the infin- 
ite system of stochastic differential equations in which equations (13) 
and (14) are imbedded. Each provides its own perspective on the common con- 
clusion: the exact optimum filter is infinite-dimensional and therefore com- 

putationally infeasible. Our interest then turns naturally to suboptimum but 
practically implementable filters and, with it, what each of the three 
approaches offers in suggesting suboptimum estimator structures and design 
procedures and in providing a basis for error analysis (performance evalua- 
tion) . Before turning to this, however, we review briefly the special case of 
a linear system, for which the optimum filter happens to be finite-dimensional. 
Apart from its intrinsic interest as a filtering problem with an exact, imple- 
mentable solution, the linear case serves both as a benchmark and as a source 
of potential structures for suboptimal candidate designs. 


IV . LINEAR FILTERING PROBLEMS 


When the system (1) is linear, 

f(x,t) = A(t)x , g(x,t) = C(t)x (15) 

the conditional mean and covariance can be generated from the finite- 
dimensional system: 

dX|. = A(t)x^. dt + E (t)C’ (t)vr^ (t) [dz - C(t)x^. dt] ; Xq = Xq (16) 

z:(t) = A(t)Z(t) + Z:(t)A'(t)- Z:(t)C'(t)Wkt)C(t)2(t)+D(t)D'(t); 2(0) * Zg 

(17) 


The filter defined by equations (16) and (17) is known as the Kalman (or 
Kalman-Bucy) filter. It can be derived from the general equations (13) 
and (14) in x^ and 2^, from the Bucy representation (11), or directly using 
any of a variety of techniques such as the projection theorem. The simplifi- 
cation of equations (13) and (14) to (16) and (17) consists essentially of 
observing that when f and g are linear they commute with conditional expec- 
tation. Not only are no additional equations required beyond equations (16) 
and (17) to completely specify the conditional mean and covariance, but 


170 



equation (17) for the conditional covariance does not depend on the observa- 
tion process z. Thus Z is nonrandom and can be precomputed. To emphasize 
this, we write Z(t) rather than Z^ in this case. 

The linearity of the system and the Gaussianness of the noise processes 
results in the state and observation processes being jointly Gaussian, so that 
the conditional density of the state given the past observations is also 

Gaussian. This Gaussian posterior density is thus completely described by the 
conditional mean and the conditional covariance Z^.. Furthermore, the 

conditional mean is a linear function of the data Z and, in fact, is the 
finite-dimensional linear fxmction of the data given by equation (16) . For 
the model equation (la") that includes a control causally derived from the 
observations with f(x,u,t) = A(t)x + B(t)u and g(x,t) = C(t)x, the state 
and observation processes are no longer Gaussian, but the conditional density 
of the state given past observations and controls is still Gaussian with con- 
ditional mean x^, the finite- dimensional linear function of past u and z, 
given by 

dx^ = A(t)x^ dt + B(t)u^ dt + Z (t) C ' (t)W”^ (t) [dz^ - C(t)x^ dt] (18) 

The conditional covariance is still given by equation (17) and remains precom- 
putable and independent of u and z (this fact is of major importance in sto- 
chastic control situations) . 

Finally, we note the simple structure of the linear filter (18). As 
noted, for example, in reference 27, it consists of a model of the determinis- 
tic part of the system (la") (which generates, in particular, the first two 
terms on the right side of eq. (8)), a unity gain negative feedback loop that 
generates the innovations dz - Cx^ dt, and a gain ZC*W^^ that operates on 
the innovations to produce the third term on the right side of equation (18) . 
Imitation of this simple structure is one possible starting point in develop- 
ing approximate filters for nonlinear problems. 


V. RELINEARIZED ESTIMATORS AND EXTENDED KALMAN FILTERS 


Because linear estimation problems are so amenable to analysis and have 
finite- dimensional solutions, while nonlinear problems require infinite- 
dimensional filters, it is natural to attempt to approximate nonlinear prob- 
lems by linear ones in the hope that the optimum filter for the linearized 
problem will suitably approximate the optimum nonlinear filter. In this vein, 
the most common approach is to continually relinearize the system equa- 
tions (1) about the current state estimate x^ using a Taylor series expan- 
sion in which second- and higher-order terms are neglected, namely, 

f(x^,t) = f(Xj.,t) + fj^(x^,t)[x^ - Xj.] (19a) 

g(Xj.,t) = g(x^.,t) + gjj(x^,t) [x^. - x^.] (19b) 
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where denotes the partial derivative of f with respect to its 

first argument evaluated at Xj-. The system equations can then be written in 
the linear form 

dXf = fjj(x^.,t)x^. dt + f^ dt + D(t)dv^ (20a) 

dZj. = + St ®(t)dw^ (20b) 

where the known (given Xj.) quantities f^- and g^ are given by 

f(. = f(x^,t) - fjj(xj.,t)xj. , gj. = g(Xj.,t) - g(x^,t)x^ 

Then, with appropriate modifications to reflect that the linear system of 
interest is now equation (20), the Kalman filter (eqs. (16) and (17)) becomes, 
after collect-terms, 

dx^ * f(x^,t)dt + S^g^(Xj.,t)Vr^(t) [dz^ - g(x^,t)dt] (21) 

Et * fx(Xt,t)Zj. + i:^f^(x^,t) + D(t)D'(t) - 2^g^(x^,t)W"l(t)gj^(x^.,t)E^ (22) 

Note that the error covariance now depends on the observation process via 

the dependence of and on x. The suboptimum estimator (eqs. (21) 
and (22)) is usually called an extended Kalman filter. 

Alternatively, instead of identifying the linearized system (20), one can 
simply substitute equation (19) into (13) and (14) for the mean and covariance 
of the nonlinear problem. The resulting equations can be simplified to equa- 
tions (21) and (22) , provided it is assumed that the suboptimum estimator x 
satisfies E[x^. - x^. j Z^] = 0 so that, for example, we have, from equa- 
tion (19a), f^ = f(x^,t) = f(xt,t). By first identifying the linearized sys- 
tem (20), the need to make this explicit assumption is avoided. Yet another 
alternative is to take the structure of the optimum linear estimator as a 
starting point, substituting instead a model of the nonlinear system to obtain 

dx^. = f(x^,t)dt + L^[dz^ - g(x^,t)dt] (23) 

where the gain L^. is yet to be determined. Subtracting equation (23) from 
(1), using the approximations (19), and writing x^ for x^. - x^. yields 

dx^. = D(t)dv^ - Lj. dw^ - L^g^(x^.,t)x^ dt 

and, at least for f^ and g^. regarded simply as time functions, the gain L^- 
that minimizes the covariance of x is identified from linear system theory 
as L^. * 2I^gx(xt , t)Vr^ (t) , thus again leading to the extended Kalman filter 
(eqs. (21) and (22)). 

The degree to which the suboptimum estimate x^. given by the extended 
Kalman filter approximates the optimum (conditional mean) is obviously related 
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to the extent to which the second- and higher-order terms in the Taylor series 
expwsion are negligible. The quality of this lat^ 3r approximation will be 
improved when the nonlinearities are small and when the deviations of the 
state and observations (from x and z) are small, that is, when Eq and the 
noise covariances DD* and GG* = W are small. ’ wever, almost nothing is 
known analytically about the performance or quality of approximation of 
extended Kalman filters (exact evaluation requires an infinite-dimensional 
calculation) and performance characteristics as mean-square error, sensitiv- 
ity, etc., are evaluated in specific cases by simulation and Monte Carlo tech- 
niques. These simulation studies are usually limited to comparisons of com- 
peting suboptimal designs, rather than relating the suboptimum estimator to 
the optimum. Bias errors (Ex ^ 0) and divergence problems are not infre- 
quently encountered. Remedies for these tend to be limited to ad hoc rules of 
thumb (e.g. , a trial- and-error Increase of DD* in the simulations, which has 
the effect of giving more relative weight to the more recent observations) . 

Various alternatives to continual relinearization about the current state 
estimate are possible. One is to linearize about a prechosen nominal trajec- 
tory; this leads to a precomputable filter gain in equation (23) and pre- 

computable Z, but performance remains difficult to assess, and can be 
expected to be worse than for continually relinearized estimators. A second 
possibility is to retain second-order (or even more) terms in the Taylor 
series expansion about x, which on substitution into equations (13) and (14) 
and simplification yields estimators considerably more complicated than equa- 
tions (21) and (22) . Again, performance evaluation is limited to simulation 
and Monte Carlo techniques (ref. 28). Other possibilities include statistical 
linearization (refs. 3 and 13) and a wide-sense approach in which certain 
nonlinear problems are converted via transformation into a linear problem with 
non-Gaussian initial state (ref. 10) . Like the extended Kalman filter 
(eqs. (21) and (22)), these estimators all suffer from the absence of analyti- 
cal methods for design, performance evaluation, error analysis, sensitivity 
studies, etc., and these characteristics must be evaluated by simulation. 
Comparisons between competing design procedures are severely limited by the 
absence of any small collection of accepted benchmark problems that might form 
a standard of comparison in simulation studies. Finally, any comparisons are 
almost always between competing suboptimum estimators and not between subopti- 
mum and optimum. 


VI. APPROXIMATIONS TO | Z^.) 

Another approach to approximate nonlinear filtering is to approximate the 
conditional density Pt^lEt I ^t^ terms of a finite number of basis func- 

tions. Expansions used here include "point mass," Fourier, Hermite polyno- 
mials, spline functions, and sums of Gaussian densities (refs. 29-35). These 
are usxially done for discrete-time problems and typically use a version of the 
Bayes rule derived from the Bucy representation theorem. Because the effects 
of truncation and other questions related to quality of approximation have not 
yielded to analytic treatment, Monte Carlo methods are used to assess filter 
perfomnance, data-processlng rates, the number of terms needed, sensitivity, 
etc. Different expansions yield different data-processing rates and different 
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estimation accuracies; what works well for one problem may not work well for 
another. Computational effort is typically large. Comparison studies have 
been limited to Monte Carlo comparisons between suboptimal estimators; almost 
nothing is known as to how the performance of the suboptimal filters compares 
to the optimum achievable performance. . 

VII. PERFORMANCE BOUNDS 


The infinite-dimensionality of the optimum nonlinear filter, on the one 
hand, and the difficulty in evaluating the performance of extended Kalman 
filters and other suboptimum estimators on the other makes it desirable to 
seek out an analytical framework within which suboptimum filters can be 
designed and their performance evaluated with realistic computational effort. 
One direction along these lines which has received attention is an analysis in 
terms of precomputable upper and lower bounds. To establish bounds, some 
explicit assumptions are needed on the functions f and g in equations (la) or 
(la”) and (lb) , and much of the work here considers systems incrementally 
conic (or cone-bounded) in the sense that 

f(x + 6iU + Y>t) - f(x,u,t) = A(t)x + B(t)u + ^^(t) + £ 2 ^^) 
g(x + 6,t) - g(x,t) = C(t)x + 

llejl ^ a(t)||6||. Ilejl <b(t)|M|. IIS 3 II i c(t)||6|| 

Thus incrementally conic systems are linear to within a uniformly Lipschltz 
residual. Cone parameters A, B, and C define a nominal linear system, while 
a, b, and c measure the degree of nonlinearity. Many common modeling func- 
tions are incrementally conic, including sinusoids, inverse tangents, and 
saturating nonlinearities; also included are linear systems whose parameters 
are known only to within specified tolerances. Polynomials and exponentials 
in X and u are not incrementally conic. 

Within the framework of incrementally conic systems, upper bounds have 
been derived in reference 36 on the mean-square error associated with the 
parametrized (by K) family of easily implemented estimators: 

dx^ = f (x^,Uj.,t)dt + K(t) [dz^ - g(xj.,t)dt] (24) 

These are matrix-ordering bounds of the form 

E[(xt - x^)(x^ - Xj.)*] < 

where the n x n matrix PRCt) satisfies a linear differential equation. 

Thus, for any given K(t) in equation (24), the corresponding mean-square 
estimation error is guaranteed to be less than Pj^(t) . Furthermore, there 
exists. a choice K*(t) of K such that the corresponding upper bound 
P*(t) = Pu*(t) is at all times smaller in the matrix ordering sense than that 
for any other K(t) (i.e. , P*(t) < PR(t) for all K and all t). Because 
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K*(t) depends on the corresponding P*(t), P* satisfies a Riccati-like 
equation (just as the optimum gain for linear estimation problems depends on 
the corresponding error covariance E(t), which results in S(t) satisfying 
a Riccati equation). This does not say that the actual mean-square error of 
the estimator (1) is smaller for K* than for any other K. ^^at it does say 
is that when K* is used the guaranteed performance is smaller than the per- 
formance guaranteed using any other K. 

Under certain additional technical assumptions, it is also possible to 
derive a lower bound on the expected error covariance that can be achieved by 
any causal filter, including the optimum. This matrix-ordering bound takes 
the form (ref. 37) 

E{cov[x^ I Z^]} > [1 - r(t)]P(t) r (26) 

where P(t) is the optimum error covariance for the nominal linear estimation 
problem (and thus depends only on A, C, Sq and the noise coyariances DD* 
and W), while the ''shrinkage" factor r(t) < 1 depends on a and c as well. 

Both upper and lower bounds share most of the properties of the optimum 
error covariance for linear problems: in particular, they are easily computed 

by contemporary standards, and they are control- law independent. The lower 
bound converges uniformly and with finite ultimate to the optimum error covar- 
iance P of the nominal linear problem as the nonlinearity vanishes 
(a,c 0); the upper bound converges similarly for reasonable choice of K, 
including the bound minimal gain K* and the optimum gain for the nominal 
linear problem. Together, the upper and lower bounds define a range in which 
must lie both the true optimum performance and the actual mean-square error of 
the suboptimum estimator (eq. (24)). In some specific cases, this range may 
be sufficiently small that the suboptimum estimator can be deemed sufficiently 
close to optimum. In others, the guaranteed performance provided by the upper 
bound might meet acceptable performance specifications with the optimum 
achievable performance being irrelevant. In such cases, the bounds provide a 
simple analytical and computational framework for design and evaluation- If 
the guaranteed performance is inadequate (but still potentially achievable in 
view of the lower bound) , one can then turn to alternative suboptimum designs 
such as those discussed earlier, with performance assessment by simulation and 
design modification by trial and error. Similar upper bounds can be found for 
extended Kalman filters (where K can depend on x as well as t) , but these 
upper bounds are larger than those where K depends only on t. The upper 
and lower bounds can be used to derive performance bounds for stochastic con- 
trol problems (ref. 38); they have also been extended to smoothing and predic- 
tion (ref. 12). 

Other lower bounds of a Cramer-Rao type have been derived in refer- 
ences 39 to 41. Upper bounds for certain rather restricted classes of systems 
can be found in reference 42 and references cited in reference 36. 
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A SURVEY OF DESIGN METHODS FOR FAILURE DETECTION 


IN DYNAMIC SYSTEMS* 

Alan S. Willsky 

Massachusetts Institute of Technology 


SUMMARY 


In this paper, we survey a number of methods for detecting abrupt changes 
c (such as failures) in stochastic dynamical systems. We concentrate on the 
class of linear systems, but the basic concepts, if not the detailed analyses, 
carry over to other classes of systems. The methods surveyed range from the 
design of specific failure-sensitive filters, to the use of statistical tests 
on filter innovations, to the development of jump process formulations. Trade- 
offs in complexity versus performance are discussed. 


I. INTRODUCTION 


With the increasing availability and decreasing cost of digital hardware 
and software, there has developed a desire in several disciplines for the 
development of sophisticated digital system design techniques that can 
greatly improve overall system performance. A good example of this can be 
found in the field of digital aircraft control (see, e.g., refs. 1-3), where 
a great deal of effort is being put into the design of aircraft with reduced 
static stability, flexible wings, etc. Such vehicles can provide improved 
performance in terms of drag reduction and decreased fuel consumption, but 
they also require sophisticated control systems to deal with problems such 
as active control of unstable aircraft, suppression of flutter, detection of 
system failures, and management of system redundancy. The demands on such a 
control system are beyond the capabilities of conventional aircraft control- 
system design techniques, and the use of digital techniques is essential. 

Another example can be found in the field of electrocardiography. In 
recent years, a great deal of effort has been devoted to the development of 
digital techniques for the automatic diagnosis of electrocardiograms (EGG) 
(see, e.g., ref. 4). Such systems can be for preliminary screening of large 
numbers of ECG*s, for monitoring patients in a hospital, etc. 


*This research was supported in part by NASA Ames Research Center under 
Grant NGL-22-009-124 and in part by NASA Langley Research Center under 
Grant NSG-1112. 
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In this paper, we review some of the recent work in one area of system 
theory that is important in both of these examples, as well as in many other 
system design problems. Specifically, we discuss the problem of detecting 
abrupt changes in dynamical systems. In the aircraft control problem, one is 
concerned with the detection of actuator and sensor failures, while in the ECG 
analysis problem, one wants to detect arrhythmias — sudden changes in the 
rhythm of the heart. For simplicity in our discussion, we refer to all such 
abrupt changes as "failures,” although, as in the ECG example, the abrupt 
change need not be a physical failure. Our aim in this survey is to provide 
an overview of a number of the basic concepts in failure detection. The pro- 
blem of system reorganization subsequent to the detection of a failure is con- 
sidered in several of the references. We will point out these references in 
the sequel, but we will concentrate primarily on the detection problem. 

The design of failure detection systems involves the consideration of 
several Issues. One is usually interested in designing a system that will 
respond rapidly when a failure occurs; however, in high-performance systems, 
one often cannot tolerate significant degradation in performance during 
normal system operation. These two considerations are usually in conflict. 
That is, a system designed to respond quickly to certain abrupt changes must 
necessarily be sensitive to certain high-frequency effects, and this, in 
turn, will tend to increase the sensitivity of the system to noise (via the 
occurrence of false alarms signaled by the failure detection system). The 
tradeoff between these design issues is best studied in the context of a 
specific example in which the costs of the various tradeoffs can be assessed. 
For example, one might be more willing to tolerate false alarms in a highly 
redundant system configuration than in a system without substantial backup 
capabilities. 

Generally, one would like to design a failure detection system that takes 
system redundancy into account. For example, in a system containing several 
backup subsystems, we may be able to devise a simple detection algorithm that 
is easily implemented but yields only moderate false-alarm rates. On the 
other hand, by implementing a more complex failure detection algorithm that 
takes careful account of system dynamics, one may be able to reduce require- 
ments for costly hardware redundancy. 

In addition to taking hardware issues into consideration, the designer 
of failure detection systems should consider the issue of computational 
complexity. One clearly needs a scheme that has reasonable storage and time 
requirements. It would also be useful to have a design methodology that 
admits a range of implementations, allowing a tradeoff study of system com- 
plexity versus performance. In addition, it would be desirable to have a 
design that takes advantage of new computer capabilities and structures 
(e.g., designs amenable to modular or parallel implementations). 

In this paper, we survey a variety of failure detection methods and, 
keeping the issues mentioned above in mind, we will comment on the character- 
istics, advantages, disadvantages, and tradeoffs involved in the various 
techniques. To provide this survey with some organization and to point out 
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some of the key concepts in failure detection system design, we have defined 
several categories of failure detection systems and have placed the designs 
we have collected into these groups. Clearly, such a grouping can only be a 
rough approximation, and we caution the reader against drawing too much of an 
Inference about individual designs based on our classification of them 
(several of the techniques could easily fall into a nvimber of our classes) . 

In addition, for brevity, we have limited our detailed discussions to only a 
f^ of the many techniques. Our choice of those techniques has been motivated 
by a desire to span the range of available methods and by our familiarity 
with certain of these algorithms. Finally, we have attempted to collect all 
of those studies of the failure detection problem of which we are aware, and 
we apologize for any oversights. 


II. FORMULATIONS OF FAILURE DETECTION PROBLEM 


In this paper, we are mostly concerned with the analysis of linear 
stochastic models in the standard state space form: 

System dynamics: 


x(k + 1) = $(k)x(k) + B(k)u(k) + w(k) 


Sensor equation: 


z(k) = H(k)x(k) + J(k)u(k) + v(k) 


( 1 ) 

( 2 ) 


where u is a known input and w and v are zero mean, independent, white 
Gaussian sequences with covariances defined by 

E[w(k)w’(j)l = » E[v(k)v' (j)) ] = R6j^j (3) 

where is the Kronecker delta. We think of equations (1) to (3) as 

describing the "normal operation" or "no-failure" model of the system of 
Interest. If no failures occur, the optimal state estimator is given by the 


discrete Kalman filter equations (ref. 5): 

x(k + ilk) = $(k)x(kjk) + B(k)u(k) (4) 

x(kjk) = x(k|k - 1) + K(k)y(k) (5) 

Y(k) = z(k) - H(k)x(k|k- 1) - J(k)u(k) (6) 

where y is the zero mean, Gaussian innovation process and gain K is 
calculated from 

P(k+ ilk) = $(k)P(klk)$'(k) + Q (7) 

V(k) = H(k)P(k|k - l)H'(k) + R (8) 
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K(k) - P(k|k - 1)H' (k)v“^(k) (9) 

P(k|k) - P(k|k - 1) - K(k)H(k)P(k|k - 1) (10) 

Here P(l|j) is the estimation error covariance of the estimate x(i|j) 
and V(k) is the covariance of Y(k)* We refer to equations (4) to (10) as 
the ’'normal mode filter" in the sequel. 

In addition to the above estimator, one may also have a closed-loop 
control law, such as the linear law, 

u(k) = G(k)x(k|k) 

We then obtain the normal operation configuration depicted in figure 1. 

The problem of failure detection is concerned with detecting abrupt 
changes in a system, as modeled by equations (1) to (3) . Such abrupt changes 
can arise in a number of ways. For example, in aerospace applications, one 
is often concerned with the failure of control actuators and surfaces. Such 
abrupt changes can manifest themselves as shifts in the control gain matrix B, 
increased process noise, or as a bias in equation (1) (as might arise if a 
thruster developed a leak (ref. 6)). In addition, failures of sensors may 
take the form of abrupt changes in H, increases in measurement noise, or as 
biases in equation (2). For simplicity, we refer to abrupt changes in 
equation (1) as "actuator failures," and shifts in equation (2) are called 
"sensor failures." Again, we point out that, in many applications, shifts 
in equation (1) or (2) may be used to model changes in observed system behav- 
ior that have nothing to do with actuators or sensors. 

The main task of a failure detection and compensation design is to modify 
the normal mode configuration to include the capability of detecting abrupt 
changes and compensating for them by activating backup systems, adjusting the 
feedback design appropriately, etc. Conceptually, we think of the detection- 
compensation system as part of the filtering portion of the feedback loop. 

As illustrated in figures 2 and 3, the resulting filter design can take one 
of two forms: either we perform a complete redesign of the filter, replacing 

equations (4) to (10) with a filter that is sensitive to failures or we design 
a system that monitors the normal system configuration and adjusts the system 
accordingly. We will discuss examples of both structures. 

As mentioned earlier, we will concentrate primarily on the problem of 
failure detection, which we consider to consist of three tasks — alarm, 
isolation, and estimation. The alarm task simply consists of making a binary 
decision, either that something has gone wrong or that everything is fine. 

The problem of isolation is that of determining the source of the failure 
(e.g., which sensor or actuator has failed, what type of arrhythmia has 
occurred, etc.). Finally, the estimation problem involves the determination 
of the extent of failure. For example, a sensor may become completely non- 
operational (an "off" or "hard-over" failure), or it may simply suffer degrad- 
ation in the form of a bias or increased inaccuracy, which may be modeled as 
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an increase in the sensor noise covariance. In the latter case, estimates 
of the bias or the increase in noise may allow continued use of the sensor, 
albeit in a degraded mode. Clearly, the extent to which we need to perform 
these various tasks depends on the application. If a human operator is 
available, we may only be interested in generating an alarm that tells him to 
perform further tests. In other systems in which backups are available, we 
might settle for failure isolation without estimation. On the other hand, in 
the absence of hardware redundancy, we may be interested in using a degraded 
instrument and thus would need estimation information. 

Intuitively, we can associate increased software system complexity with 
the tasks, that is, isolation requires more sophisticated data processing 
than an alarm, and estimation more than isolation. On the other side, as we 
increase failure detection capabilities, we may be able to decrease hardware 
redundancy. Also, in some applications, we may be able to delay isolation and 
estimation until after an alarm has been sounded. In such a sequential 
structure, one increases detector complexity after a failure has been 
detected, thereby reducing the computational burden during normal operation. 
Again the details of such considerations depend on the particular application. 

Another tradeoff involving failure detection system complexity involves 
its relation to detection system performance. For example, one might expect 
that one could achieve better alarm performance by using a priori knowledge 
concerning likely failure modes. That is, by looking for specific forms of 
system behavior characteristic of certain failures, one should be able to 
improve detection performance. Thus it seems likely that alarm performance 
(as measured by the tradeoff between false alarms and missed detections) 
will be improved if we attempt simultaneous detection. Isolation, and estima- 
tion of failures. This tradeoff of complexity versus performance is extremely 
important in the design of failure detection systems. 

In the following sections, we discuss several failure detection methods 
and comment on their characteristics with respect to the issues mentioned in 
this and the preceding section. We have not provided a general set of fail- 
ure models to be considered, as the various techniques are based on quite 
different failure models. These are described as we discuss the various 
methodologies . 


Ill . "FAILURE-SENSITIVE" FILTERS 


Our first class of failure detection concepts is aimed at overcoming 
the problem of an "oblivious filter." As has been noted by many authors 
(refs. 5 and 7-9), the optimal filter defined by equations (4) to (10) 
performs well if there are no modeling errors. However, it is possible for 
the filter estimate to diverge if there are substantial unmodeled phenomena. 
The problem occurs because the filter "learns the state too well," that is, 
the precomputed error covariance P and filter gain K become small, and 
the filter relies on old measurements for its estimates and is oblivious to 
new measurements. Thus, if an abrupt change occurs, the filter will respond 
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quite sluggishly, yielding poor performance. Consequently, one would like 
to devise filter designs that remain sensitive to new data so that abrupt 
changes will be reflected in the filter behavior. 

Two well-known techniques for keeping the filter sensitive to new data 
are the exponentially age-weighted filter studied by Fagih (ref. 7) and by 
Tam and Zaborszky (ref. 8) and the limited memory filter proposed by 
Jazwinski (ref. 9). Others, such as increasing noise covariances or simply 
fixing the filter gain, are discussed by Jazwinski (ref. 5). These techniqties 
yield only Indirect failure information; that is, if an abrupt change occurs, 
these filters will respond faster than the normal filter, and one can base 
a failure detection decision on sudden changes in x. 

It is Important to note a performance tradeoff evident in this method. 

As we Increase our sensitivity to new data (by effectively increasing the 
bandwidth of the Kalman filter) , our system becomes more sensitive to sensor 
noise, and the performance of the filter under no-fallure conditions degrades. 
In some cases, this can be rather severe, and one may not be able to tolerate 
the degradation in overall system perfomance under no-failure conditions. 

One might then consider a two filter system — the normal mode filter (eqs. (4)- 
(10)) as the primary filter, with this type of failure-sensitive, filter as an 
auxiliary monitor, used only to detect abrupt changes. We remark that the 
tradeoff between detection performance and filter behavior under normal 
conditions is a characteristic of all failure detection systems and is 
analogous to the costs associated with false alarms and missed detections in 
standard detection problems (ref. 10). 

The techniques mentioned so far are rather indirect failure detection 
approaches. Several methods have been developed for the design of filters 
sensitive to specific failures. One method Involves the inclusion of several 
"failure states" in the dynamic model (eqs. (l)-(3)). Kerr (ref. 11) has 
considered a procedure in which failure modes, such as the onset of biases, 
are Included as state variables. If the estimates of these variables vary 
markedly from their nominal values, a failure is declared. A two-confidence 
interval overlap decision rule for failure detection using such failure states 
is described and its performance is analyzed in reference 11 ._ Note that this 
approach provides failure isolation and estimation at the expense of Increased 
dimensionality and some performance degradation under no-fallure conditions 
(inclusion of the added states effectively opens up the bandwidth of the 
Kalman filter) . 

An alternative to the addition of failure states to the dynamic model 
is the class of detector filters developed by Beard (ref. 12) and Jones 
(ref. 13). Their work has led to a systematic design procedure for detecting 
a wide variety of abrupt changes in linear time-invariant systems. They 
consider the continuous-time, time- invariant, deterministic system model: 


x(t) = Ax(t) + Bu(t) 

(11) 

z(t) = Cx(t) 

(12) 
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and design a filter of the foirm 

x(t) “ Ax(t) + D[z(t) - Cx(t)] + Bu(t) (13) 

The primary criterion In the choice of the gain matrix D Is not that 
equation (13) provide a good estimate of x (as It Is with observers or 
optimal estimators ) , but rather that the effects of certain failures are 
accentuated In the filter residual: 

Y(t) - z(t) - Cx(t) (14) 

The basic Idea Is to choose D so that particular failure modes manifest 
themselves as residuals that remain In a fixed direction or In a fixed plane. 

To Illustrate the Beard -Jones approach, consider a simple example from 
reference 12. Suppose we wish to detect a failure of the 1th actuator 
(l.e.. In the actuator driven by the 1th component of u) . If we assume the 

failure takes the form of a constant bias, our state equation becomes 

x(t) * Ax(t) + B[u(t) + ve^] 

“ Ax(t) + Bu(t) + vb^ , t > tQ (15) 

where e^ Is the 1th standard basis vector, b^ Is the 1th column of B, 

and tQ Is the (unknown) time of failure. Suppose we consider the case of 
full state measurement — let C = I. In this case, we obtain a differential, 
equation for the residual 

Y(t) - [A - D]Y(t) + vb^ (16) 

If we choose D = oI + A, we obtain 


Y(t) - -aY(t) + vb^ 


Y(t) = e 


-a(t-tj 


-ot, 


0 Y(tn) 




(17) 


Thus, as the effect of the initial condition dies out, y(t) maintains a 
fixed direction (b^) with magnitude proportional to failure size (v) . 

Note that as we increase a (thus increasing filter gain) , the initial 
condition dies out faster, but the magnitude of the steady-state value of y 
decreases. Thus, if there is any noise in the system, we cannot make a 
arbitrarily large. 

In their work. Beard and Jones consider the design of such filters for 
an extremely wide variety of failure modes, including actuator and sensor 
shifts and shifts in A and B. The initial deterministic analysis for all 
of these cases was considered by Beard (ref. 12), while a systematic design 
procedure is given by Jones (ref. 13) for the design of gain D to allow 
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detection of several failure modes. Jones’ approach is quite geometric in 
nature » and his formulation allows one to gain considerable insight into thh 
detection problem. As pointed out in reference 13, the gain selection problem 
is; quite similar to the output decoupling problem and requires the introduc- 
tion of the Important concept of "mutually detectable failure modes" to answer 
the question of whether one can simultaneously distinguish between several 
types of failures. Thus the question of failure isolation is of central 
Importance in the design methodology derived in reference 13. 

The results in references 12 and 13 represent perhaps the most thorough 
study of the basic concepts underlying failure detection. The tradeoff 
between detection and filter performance is discussed in depth in reference 13 
and an attempt is made in reference 12 to Introduce the concept of the level 
of redundancy in a dynamical system. 

As mentioned in the example, the basic design procedure is deterministic. 
However, in this simple example, we can see how one can take noise into 
account. If the system (eqs. (11) and (12)) contains noise, we have seen 
that one may not wish to make the scalar a as large as possible. In fact, 
one could choose a so as to minimize the mean-square estimation error in 
the detector filter when there is no failure. In his thesis, Jones (ref. 13) 
describes a procedure in which one first chooses the structure of D for 
failure detection purposes and then chooses the remaining free parameters 
to minimize the estimation error covariance. Although this yields a subopti- 
mal filter design, it may work quite well, as it did in the problem reported 
in reference 13. 

The Jones- Beard design methodology is extremely useful conceptually, 
it can be used to detect a wide variety of failures, and it provides detailed 
failure isolation information. It is suboptlmal as an estimator, and if 
this presents a serious problem, one might wish to use the detector filter as 
an auxiliary monitoring system. This appears to be only a minor drawback, 
and the major limitation of the approach is its applicability only to time- 
invariant systems. 


IV. VOTING SYSTEMS 


Voting techniques are often useful in systems that possess a high degree 
of parallel hardware redundancy. Memoryless voting methods can work quite 
well for the detection of "hard" or large failures, and Gilmore and McKern 
(ref. lA), Pejsa (ref. 15), and Ephgrave (ref. 16) discuss the successful 
application of voting techniques to the detection of hard gyro failures in 
inertial navigation systems. 

In standard voting schemes, one has (at least) three identical instru- 
ments. Simple logic is then developed to detect failures and eliminate 
faulty instruments, for example, if one of the three redundant signals 
differs markedly from the other two, the differing signal is eliminated. 
Recently, Broen (ref. 17) has developed a class of voter estimators that 
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possesses advantages relative to standard voting techniques. Consider the 
dynamical system 


x(k + 1) « $x(k) 

with a triply redundant set of sensors: 

yj(k) = HjX(k) + Vj(k) 

y2<k) = H^xCk) + v^Ck) 

y3<k) - HgX(k) + Vg(k) 


( 18 ) 


( 19 ) 


Broen develops a set of recursive filter equations for computing the estimate 
x(k) that minimizes 



3 


E 

j=i 


WjiY’( 1 )R-'yj( 1 ) 


(20) 


where Rj is the covariance of the measurement noise Vj and Yj Is the 
Innovations sequence 

Yj( 1 ) = y^d) - Hj»^‘^x(k) ( 21 ) 

Here is a function of (i) , 72(1), and 73(1) which is large if Yj (i) 

is close to the other two Ytn(i) terms and is small if Yj(i) deviates 
greatly from the other two. In this way, one obtains a "soft" voting pro- 
cedure in which faulty sensors are smoothly removed from consideration. This 
greatly alleviates the cost of false alarms, but the price is the on-line 
computation of the filter gain (which is a function of the w..). Note that 
in equation ( 19 ) , Broen appears to allow the y^ to be physically different 
sensors (different terms) , but the analysis of his paper makes it clear 

that he requires identical sensors (i.e., = ^3)- 


Voting schemes are generally relatively easy to implement and usually 
provide fast detection of hard failures, but they are applicable only in 
systems that possess a high level of parallel redundancy. They do not 
generally take advantage of redxmdant information provided by unlike sensors, 
and thus cannot detect failures in single or even doubly-redundant sensors. 

In addition, voting techniques can have difficulties in detecting "soft" 
failures (such as a small bias shift) . 


V. MULTIPLE HYPOTHESIS FILTER DETECTORS 


A rather large class of adaptive estimation and failure detection schemes 
Involves the use of a "bank" of linear filters based on different hypotheses 
concerning the underlying system behavior. In the work of Athans and Wlllner 
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(ref. 18) and Lainlotls (ref. 19), several different sets of system matrices 
are hypothesized. Filters for each model are constructed, and the innovations 
from the various filters are used to compute the conditional probability that 
each system model is the correct one. In this manner, one can do simultaneous 
system identification and state estimation. In addition, an abrupt change 
in the probabilities can be used to detect changes in true system behavior. 
This technique has been investigated in the context of the adaptive control 
of the F-8C digital fly-by-wire aircraft by Athans et at. (ref. 20) and also 
has been applied to the problem of classifying rhythms and detecting rhythm 
shifts in electrocardiograms. Extremely good results in the latter case are 
reported by Gustafson et at. (ref. 21). 

Techniques involving multiple hypotheses have also been used to design 
failure detection systems. Montgomery et at. (refs. 22 and 23) have used such 
a technique for digital flight-control systems and have studied its robustness 
in the presence of nonlinearities via simulations. Recently, a technique 
involving a bank of observers has been devised (ref. 24), and a successful 
application to a hydrofoil sensor failure problem is reported by Clark et at. 
(ref. 24). Also, Willsky et at. (refs. 25 and 26) have applied the methodol- 
ogy devised by Buxbaum and Haddad (ref. 27) to study failure detection for an 
inertial navigation problem. We will briefly describe this technique to 
illustrate some of the concepts underlying the bank-of-f liters approach. We 
also refer the reader to Wernersson (ref. 28) for a technique similar to that 
discussed in reference 26 . 

Consider the system 


x(k + 1) = $(k)x(k) + w(k) (22) 

z(k) = H(k)x(k) + v(k) (23) 

We are interested in detecting sudden shifts in certain of the components 
of X (e.g., bias states). We model these shifts by choosing the distribu- 
tion of w appropriately. Let {f 2 ,..., f^.} be the set of hypothesized 
failure directions . We then assume that w has a high probability of being 
the usual process noise and a small probability of including a burst of noise 
in each of the failure directions. Thus the density for w(k) is 

PqN( 0,Q) + f' p.N(0,Q + off') (24) 

i*l 1 1 

r 

p = 1, Pn » p. i = l,...,r (25) 

i=0 u X 


Here N(m,P) is a normal density with mean m and covariance P. 

If we hypothesize such a density at each point in time and if we assume 
that x(0) is normally distributed, we have the following expression for the 
conditional density of x(k) , given z(l) z(k): 
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r 



(26) 


Here' _i = (Iq , . , . and the density has the following interpretation. 

Let = (j Q » • . . » jk_l) be a random k-tuple where jg = i if there is a 
shift in the f. direction at time s (1 = 0 is used to denote no shift). 
Then 


and 


and Pj 


J1 ^ ^ 

assuming j “ (l.e., assuming 


Pi = Pr[j^ = i|z(l) z(k)] (27) 

are the mean and covariance of the Kalman filter designed 
w(s) 


has covariance Q + Oj fj f\ ). The 

S '*’S 

JEi terms can be computed sequentially as a function of the various filter 
innovations. We refer the reader to references 25 to 27 for the details of 
the calculations. 


Note that the implementation of equation (26) requires an exponentially 
growing bank of filters (there are (r+1)^ terms in eq. (26)). To avoid this 
problem, a number of approximation techniques have been proposed (refs. 25-27). 
The one used in reference 26 involves hypothesizing shifts only once every N 
steps. At the end of each N step period, we "fuse" the (r+1) densities 
into a single density and begin the procedure again. In this way, we imple- 
ment only (r+1) filters at any time. Note that the techniques devised in 
references 18, 19, and 22 do not involve growing banks of filters (as the 
number of hypothesized models do not grow in time). However, it is possible 
for all filters in the bank to become oblivious, and thus shifts between the 
hypotheses may go undetected (see refs. 21 and 26 for examples). The 
technique of periodic fusing of the densities and Initiation of new banks 
effectively avoids this problem (as would designing the original bank using 
age-weighted filtering techniques). 


The technique described above was applied to the problem of detecting 
gyro and accelerometer bias shifts in a time-varying inertial calibration and 
alignment system. The results of these tests are extremely impressive. 

This is not surprising as the multiple-hypothesis method computes precisely 
the quantities of interest — the probabilities of all types of failures under 
consideration. The cost associated with such a high level of performance is 
an extremely complex failure detection system. Note, however, that the paral- 
lel structure of the system allows one to consider highly efficient parallel 
processing computer implementations. In addition, the use of reduced-order 
filters for the various failure hypotheses may increase the practicality of 
such a scheme, or one might consider the use of a simpler detection-only 
system to detect failures, with a switch to a multiple hypothesis procedure 
for failure isolation and estimation after a failure has been detected. 


However, even if such a failure detection scheme cannot be implemented 
in a particular application, it provides a useful benchmark for comparison 
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with simpler techniques. In addition, by studying the simulation of a 
multiple hypothesis method, one can gain useful insight into the dynamics 
of failure propagation and detection (see the discussion in ref. 26). 

McCarty (ref. 29) has developed a method for rejecting bad measurements 
which bears some similarity to the approach just discussed. Each measurement 
has a binary random variable g(k) associated with it. If g(k) =* 1, the 
measurement is "good" (i.e., the measurement contains the signal of Interest), 
while g(k) = 0 denotes a bad data point (measurement is pure noise). 

McCarty devises a maximum likelihood approach for estimating the values of 
the exponentially growing set of possibilities [g(i) = 1 or 0, i = l,...,k]. 
He also allows these variables to have a sequential correlation (i.e., knowing 
that the present measurement is good or bad says something about the next 
observation) • A computationally feasible approximation method is devised and 
simulation results are described (see ref. 29 for details). 

Recently, Athans et at* (ref. 30) also considered the problem of 
designing an estimator that can detect and remove bad or false measurements. 
Their approach is Bayesian in nature, that is, an estimate is generated of the 
a posteriori probability that a given measurement is false. The method of 
calculation of these pseudo^-probabilities is quite similar to that used in the 
other multiple hypothesis methods (see refs. 18, 19, 22, 23, and 27). (The 
reader is referred to ref. 30 for details of the analysis and for a dis- 
cussion of some successful simulation results.) 


VI. JUMP PROCESS FORMULATIONS 


The problem of detecting abrupt changes in dynamical systems suggests 
the use of jump process techniques in devising system design methodologies 
(see refs. 31-33 for general results on jump processes). One models 
potential failures as jumps, characterized by a priori distributions that 
reflect initial information concerning failure rates. The size of the possible 
failures are usually taken to be known. One could, however, model failure 
magnitude as a random variable. This leads to a compound jump process formu- 
lation that greatly complicates the desired analysis. In any event, taking 
such a jump process formulation, one can devise failure-sensitive control laws 
and methods for computing the conditional probability of failure. Control 
problems of this type have received a great deal of attention in the litera- 
ture. Sworder and Robinson (refs. 34-38) and Ratner and Luenberger (ref. 39) 
have considered the design of control laws which consider the possibility of 
sudden shifts in system matrices. The results they obtained are for the full- 
state feedback problem with no system randomness other than the jumping of the 
system matrices among a finite set of possible matrices. 

Davis (ref. 40) has used nonlinear estimation techniques to solve a fault 
detection problem. His formulation is as follows: consider the scalar sto- 

chastic equations, 
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dx(t) « a(t)x(t)dt + g(t)dv(t) 

(28) 


dy(t) « h(t)x(t)dt + dw(t) 

(29) 

where 

w and V are Independent Brownian motion processes and 


t 

a(t) “ ap(t)[l - C(t)] + a^(t)C(t) 

(30) 

where 

5(t) - I ° ^ ^ ^ \ 

1 1 , t > T ) 

(31) 

and T Is a random variable. Here we Interpret a^ as the unf ailed dynamics, 
and represents the failure mode. Davis derives the optimal, Inflnlte- 

dlmenslonal equations for the computation of the conditional mean of x and 
the conditional probability 


|(t|t) - Pr[t > T|y(s) , 0 < s < t] 

(32) 


An Implementable approximation Is described In reference 40, but evaluation of 
Its performance has not yet been made. 

Note that Davis' method leads to an estimate of x that Is suboptlmal 
under no-f allure conditions. Chlen (ref. 41) has devised a jump process 
formulation that avoids this difficulty for the problem of detecting a jump 
or a ramp In a gyro bias. He considers the dynamical model: 

i(t) = wx(t) + w(t) (33) 

where w Is a white noise process. Three hypotheses are conjectured for the 
form of the gyro output: 

Normal mode H^: 

z(t) - x(t) + v(t) , Vt (34) 

Bias mode : 

z(t) = x(t) + m5(t) + v(t) , t > T (35) 

Ramp mode 

z(t) - x(t) + n(t - T)C(t) + v(t) , t > T (36) 

where n and m are unknown constants, v is white noise, T Is the time of 
failure, and C(t) Is as In equation (31). 
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Chien's approach is as follows: design a filter based on Hg (which will 

thus yield the optimal estimate for t < T, assuming no false alarms occur) 
and determine the steady-state effect of the degradations and H 2 on the 
filter residuals. If one then hypothesizes a failure rate q, that Is, 

P(T > t) * e"*^^ (37) 

and If one further assumes a nominal size for the bias m, one can then com- . . 
pute an approximate stochastic differential equation for Pr(H^|z(s), s < t), 
in which the Input to this equation is the residual y of the Hq filter. ; The 
details of the analysis are described In reference 41. 

For his problem, Chien is able to demonstrate that his detection proce- 
dure — based on the assumption of a nominal value for the bias failure m — 
has the capability of detecting biases larger than m and also can be used 
to detect ramps (mode H^) . Of course, the delay times until detection In 
these cases are greater than If one Implemented a filter based on the proper 
bias size or If one were looking for a ramp (Indicating the potential useful- 
ness of estimating the failure magnitude). The major advantages of Chien's 
approach are the simplicity of the detector (Implementation of a scalar 
stochastic equation) and the fact that one obtains an estimate of precisely 
the quantity of interest — the conditional probability of failure. The 
simplicity of the scheme may, in fact, make it a great deal more robust in 
the face of system modeling errors (such as the use of an extremely simplified 
gyro error model) than more sophisticated approaches. Also, this approach 
leads to no degradation in performance prior to detection of the failure. In 
addition, the use of a probabilistic description of the time of failure allows 
one to avoid the problem of the oblivious filter — that is, the fact that a 
failure can occur at any time has been incorporated in the design, which 
therefore will remain sensitive to new data. 

The drawbacks of the scheme are the use of a fixed bias size and the use 
of the steady-state effect of the failure on the filter residual. The first 
of these may not be too much of a problem (as Chien pointed out) , but the 
second may cause difficulties. Specifically, this limits the approach to 
time- invariant systems and filters. In addition, as the transient effect 
of the failure has been ignored, it may be difficult to make quick detections 
of certain changes (i.e., we may have to wait until the transient dies out). 

In the next section, we discuss an approach (the GLR method) that has several 
concepts in common with Chien's approach and that allows one to overcome 
these two drawbacks (at the cost of added computational complexity, of course) . 

Jump process formulations appear to be quite natural for failure detec- 
tion problems. One usually makes approximations in the analysis to obtain 
implementable solutions. These simplifications impose some limitations on 
the capabilities of the designs, but presently there is no systematic analyt- 
ical procedure for evaluating these limitations or for studying tradeoffs 
between design complexity and system performance. 
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VII. INNOVATIONS-BASED DETECTION SYSTEMS 


Chien*s failure detection technique can also be placed in the class of 
failure detection methods that involve monitoring the innovations of a filter 
based on the hypothesis of normal system operation. In such a configuration, 
the overall system uses the normal filter until the innovation monitoring 
system detects some form of aberrant behavior. The fact that the monitoring 
system can be attached to a filter-controller feedback system is particularly 
appealing since overall system behavior is not disturbed until after the 
monitor signals a failure and since the monitoring system can be designed to 
be added to an existing system. 

Mehra and Peschon (ref. 42) suggested a number of possible statistical 
tests to be performed on the innovations. One of these is a chi-squared 
test that was applied by Willsky et at. (refs. 25 and 26). Let y(h) be the 
p-dlmenslonal Innovations for the filter defined by equations (4) to (10). If 
the system is operating normally, the innovations is zero mean and white with 
known covariance V(k). In this case, the quantity 

k 

Mk) = ^ Y'(j)V^(j)Y(j) (38) 

j=k-N+l 


is a chi-squared random variable with Np degrees of freedom (refs. 25, 26, 
and 42). If a system abnormality occurs, the statistics of y change, and 
one can consider a detection rule of the form 

Failure 

Z(k) > e (39) 

failure 

With the aid of chi-squared tables, one can compute the probability Pp of 
false alarm as a function of the innovations window length N and the deci- 
sion threshold e. The probability of correct detection depends on the 

particular failure mode (see ref. 26 and the discussion of the GLR approach 
to follow). We note that, for a given failure mode, as N increases the 
probability of correct detection may decrease — that is, by averaging a 
larger number of residuals we smooth out the effect of a failure on y» and 
the detector may become somewhat oblivious (or at the very best respond quite 
slowly) to new data. On the other hand, too small a value of N may yield 
an unacceptably high value of P^,. 

The implementation of the chi-squared test (eqs. (38) and (39)) is quite 
simple, but, as one might expect, one pays for this simplicity with rather 
severe limitations on performance. As described in references 25 and 26, this 
method was applied to the same inertial calibration and alignment problem to 
which the Buxbaum-Hadded multiple hypothesis approach (refs. 25-27), described 
in section V, was applied. The performance of the chi-squared test was mixed. 
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The method Is basically an alarm method, that Is, the system (eqs. (38) and 
(39)) makes no attempt to Isolate failures, and one finds that those failure 
modes that have dramatic effects on y are detectable by this method. 

However, more subtle failures are more difficult to detect with this simple, 
scheme. Comparing the performance of the multiple hypothesis and chl-squared 
systems, we see that In some cases we can obtain superior alarm capabilities 
If we simultaneously attempt to do failure Isolation and estimation. One 
can obtain some failure Isolation Information by considering the components 
of Y separately (this may be especially useful for sensor failures), and we 
refer the reader to references 25 and 26 for a detailed discussion of this and 
other aspects of the chl-squared method. 

Another Innovation-based approach, developed by Merrill (ref. 43), Is 
motivated by a desire to suppress bad sensor data. Merrill devises a modifi- 
cation of the least-squares criterion to suppress extremely large residuals 
(which are given a very large weighting In the usual least-squares framework) , 
and he applies his methodology to a power system application. 


A final technique In this category has been studied by several research- 
ers (refs. 6 and 44-48); we will describe the most general formulation of the 
approach, developed In references 44 and 45. This technique, which we call 
the generalized likelihood ratio (GLR) approach, was In part motivated by the 
shortcomings of the simpler chl-squared procedure. The GLR approach, which 
can be applied to a wide range of actuator and sensor failures, attempts to 
Isolate different failures by using knowledge of the different effects such 
failures have on the system Innovations. The method provides an optimum 
decision rule for failure detection and provides useful failure Identifica- 
tion Information for use In system reorganization subsequent to the detection 
of a failure. In addition, one can devise a number of simplifications of the 
technique and can study analytically the tradeoff between GLR complexity and 
GLR performance. 


Consider the basic dynamical model (eqs. (1) to (3)). The following are 
four possible modifications of these equations that Incorporate certain sudden 
system changes (see refs. 21, 44, and 45 for the physical motivation for these 
and other failure modes of the same general type) : 

Dynamic jump: 

x(k + 1) « $(k)x(k) + B(k)u(k) + w(k) + q (40) 

Here v Is an unknown n-vector, 0 Is the unknown time of failure, and 
Is the Kronecker delta. Such a model can be used to model sudden shifts 
In bias states (as In the Inertial problem studied In refs. 25 and 26). 


Dynamic step : 


Here . 
ij 


x(k + 1) = $(k)x(k) + B(k)u(k) + w(k) + ^ 

Is the unit step 


(41) 
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r 1 , i > j 

1 0 , i < j 


} 


(42) 


This model can be used to model certain actuator failures (compare to the 
Beard-Jones example in section III; see eq. (15)). 


Sensor jump: 

z(k) = Hx(k) + Ju(k) + v(k) + v6^ ^ (43) 

We can use this to model bad data points. 

Sensor step: 

z(k) = Hx(k) + Ju(k) + v(k) + va^ ^ (44) 

Sudden changes in sensor biases fit into this model. 

By the linearity of the system (eqs. (1) to (3)) and the filter (eqs. (4) 
to (10)), one can determine the effect of each failure mode on the innova- 
tions. The general form is 


Y(k) = G(k;0)v + y(k) (45) 

where y(k) is the filter innovations if no failure occurs, and the matrix 
G can be precomputed (see refs. 45 and 48). This matrix, which is different 
for each of the four cases (eqs. (40) to (44)), is called the failure 
signature matrix and provides us with an explicit description of how various 
failures propagate through the system and filter. 

The full-blown GLR method involves the following: we assume we are 

looking for one of the four classes of failures and have computed the appro- 
priate signature matrix. Given the residuals, we compute the maximum likeli- 
hood estimates of v and 0 and, assuming that these estimates are correct, 
we compute the log-likelihood ratio for failure versus no failure (see "f 
ref. 10 for a general discussion of GLR methods). The implementation of the 
full GLR requires a linearly growing bank of matched filters, computing the 
best estimates of v assuming a particular value of 0e{l,...,k}. 

A number of remarks can be made concerning the GLR system. We note that, 
as with other methods such as those of Buxbaum-Haddad or Chlen, the inclusion 
of variable 0 to indicate our uncertainty as to the time of failure keeps 
the detection system sensitive to new data. However, it is the estimation of 
0 that causes the growing complexity problem. On the other hand, even if the 
full GLR is not implementable, it can serve as a benchmark for other schemes 
and can, in fact, be used as a starting point for the design of simpler 
systems. One simplification that eliminates the growing complexity is the 
restriction of the estimate of 0 to a window 
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k-N<e<k-M (45) 

where the lower bound Is Included to limit complexity and the upper bound Is 
set by failure observability and false-alarm considerations. Successful 
simulation runs with N ■= M (l.e., when we do not optimize 6 at all and have 
only one matched filter for v) are reported In reference 45. We remark only 
that the price one pays for "windowing" the estimate of 6 Is a reduction In 
the accuracy of the estimate of v. For example, for N > M, we often are 
able to detect failures extremely quickly, but if § = k - N is not the 
correct time of failure, the estimate of v may be severely degraded 
(e.g., our estimate of the slope of a ramp changes as we change our estimate 
of the time at which It started). We note that the estimation of 6 Is 
similar to time-of-ar rival estimation problems that arise in various applica- 
tions (see ref. 49 for a general discussion of several techniques). 

Note that even if the physical system and filter are time-invariant, the 
GLR monitoring system Is time-varying, as the failure signature G Includes 
transient effects. In some cases, one may be able to neglect these and use a 
simpler steady-state signature. This Is quite similar to Chlen's use (ref. 41) 
of the steady-state effect of the failure on the residuals, and the criticisms 
of that approach (given in section VI) apply here as well. 

One can also simplify the Implementation by either partially or com- 
pletely specifying the failure magnitude v. Constrained GLR (CGLR) Is based 
on the assumption that 

V = af^ (46) 

where a Is an unknown scalar and f^ Is one of r possible failure 
directions. This technique Is described in reference 45. If we completely 
specify V : 


V = Vq (47) 

we obtain the simplified GLR (SGLR) algorithm that is extremely simple to 
Implement, as we have completely eliminated the need for the matched filters 
to estimate v. The use of specified failure sizes Is similar to that 
proposed by Chien (ref. 41), although In SGLR one can use the time-varying 
failure signature, which should aid In failure detection. As Initial results 
for the detection of electrocardiogram arrhythmias Indicate (ref. 21), the 
estimation of v Is not nearly as Important for detection as matching the 
failure signatures. Also, by the use of several values of Vq (i.e., by 
implementing several parallel SGLRs) , one can achieve a high level of 
failure Isolation without a great deal of additional software complexity. In 
addition, one could consider a "dual-mode" procedure In which SGLR Is used 
for alarm and isolation, with full GLR used only afterward to estimate the 
magnitude of the failure. 
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The various simplifications of GLR, as well as full GLR, are amenable 
to certain analysis, such as the calculation of Pp, Pj^, and (at least for 
SGLR) the expected time delay in detection. By performing such analyses, 
one can study in detail the tradeoff between complexity and performance. A 
methodology for such comparisons is presently being developed and is being 
applied to an aircraft failure detection problem. Initial results are 
reported by Chow et dl* (ref. 48), and a description of a detailed methodology 
will be reported in the near future (see ref. 50). In addition to the calcu- 
lation of Pp and Pj), the comparison methodology reported in reference 50 
includes the computation of cross-detection probabilities — that is, the 
probability of detecting a failure of type A when a failure of type B has 
occurred. Such information can be useful in designing failure isolation pro- 
cedures and also in determining whether failure detector A can be successfully 
used as an alarm for failures of type B. This can lead to substantial 
simplifications in a failure alarm system. (See refs. 21, 45, and 48 for 
successful simulations of the GLR method.) 

Presently, the GLR method is being extended to other failure modes: 
hard-over actuator^ increased process noise, hard-over sensor, and increased 
sensor noise. 

Hard-over actuator failure: 

x(k + 1) = $(k)x(k) + [B + w(k) (48) 

With this model, we can take into account complete (or "off”) failures of 
certain actuators. For example, an off failure of the ith actuator can be 
modeled by choosing M all zero except for the ith column, which is taken 
to be the negative of the ith column of B. The GLR detector for equa- 
tion (48) is presently under development (refs. 48 and 50), and we note that 
this model is more difficult than the others as the effect of the failure is 
modulated by the input values u(k). 

Increased process noise failure: 

x(k + 1) = $(k)x(k) + B(k)u(k) + w(k) + 5(k)^^^ ^ (49) 

Here 5 is additional white process noise. 

Hard-over sensor failure: 

z(k) = Hx(k) + Ju(k) + v(k) + [Mx(k) + Su(k)]a^ ^ (50) 

Here the failures are modulated by u and x, and a failure of the ith 
sensor is modeled by choosing the ith rows of M and S appropriately. 

Increased sensor noise failure: 

z(k) - Hx(k) + Ju(k) + v(k) + e 
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The analysis of these failure modes Is presently being performed (refs. 48 
and 50), and It Is anticipated that SGLR algorithms will also be developed; 

In addition to these failure modes, one can develop additional models 
along these lines for particular applications. In particular, we have devel- 
oped several additional models similar to those described by equations (40) to 
(44) for our work on the detection and classification of arrhythmias In 
electrocardiograms. The results reported In reference 21 are rather striking; 
as In all the tests performed we observed no false alarms, detected all rhythm 
changes Immediately (with no Incorrect estimates of 6) , and classified all 
rhythm changes correctly. These tests used the full GLR approach and have 
provided useful Insight Into the characteristics of the method. For example, 
the use of maximum likelihood estimates of v and 6 precludes the use of 
a priori, statistics on these variables. In the EGG problem, one Is quite : 
Interested In accurate estimates of v, and one also can come up with reason- 
able a priori statistics on v based on physical arguments. Thus, It may pay 
to Incorporate such a priori statistics Into the GLR system, and this can be 
done rather easily by proper Initialization of the matched filters estimating 
V. On the other hand, for the EGG problem, one does not want to look for 
abrupt changes at one point In the record more than at another, and thus It 
does not make sense to Include a priori statistics on 6. In fact, one can 
argue that Inclusion of a priori failure Information tends to discount the 
observed data to avoid false alarms (unless failures are extremely likely) , 
and one should probably avoid the Inclusion of such Information unless one Is 
especially worried about false alarms. However, If one wishes to use such 
data, one can use the Interpretation of the likelihood ratios as ratios of 
conditional probabilities of failure times to detemlne the appropriate modi- 
fication of GLR (ref. 45). 

Finally, note that the GLR system provides extremely useful Information 
for system compensation subsequent to the detection of a failure. For 
example, one can use the GLR-produced estimates of v and 6 to determine 
an optimal update procedure for the filter estimate and covariance (ref. 45). 
Once this update has been performed, the GLR system can be used to detect 
further failures, thus allowing the detection of multiple events. (See 
refs. 45 and 48 for further discussions of the use of GLR-produced Information 
In the design of failure compensation systems.) 


VIII. CONCLUSIONS 


We have discussed a number of the Issues Involved In the design of fail- 
ure detection systems. We have also reviewed a variety of existing failure 
detection methods and have discussed their characteristics and designs trade- 
offs. The failure detection problem Is an extremely complex one, and the 
choice of an appropriate design depends heavily on the particular application. 
Issues such as available computational facilities and level of hardware 
redundancy enter in a crucial way in the design decision. For example, as we 
have mentioned, the use of a sophisticated failure detection-compensation 
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system may allow one to reduce the level of hardware redundancy without much 
loss In overall system reliability. 

The development *o£ failure detection methods Is still a relatively new 
subject. At this time, most of the work has been at a theoretical level with 
only a few real applications of techniques (refs. 6, 14-17, 21, and 23). 

Much work remains to be done In the development of Implementable systems 
complete with a variety of design tradeoffs. Work Is needed In the develop- 
ment of efficient techniques for failure compensation and system reorganiza- 
tion. In addition, there Is a great need for the analysis of the robustness 
of various failure detection systems in the presence of variations In system 
parameters and In the presence of modeling errors and system nonllnearltles. 
For example. It* Is conjectured that S6LR Is less sensitive to parameter 
errors than the more complex full GLR. However, presently there are no 
analytical results or simulations to support this conjecture. These and 
other Issues, such as the Incorporation of fault-tolerant computer concepts 
Into an overall reliable design methodology (see ref. 51) await Investigation 
in the future. 
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Figure 1.- No-fallure system configuration. 



Figure 2.- Failure detection system Involving failure-sensitive primary filter 
(€ denotes Information concerning detected failures) . 



Figure 3.- Failure detection system Involving a monitoring system for 

no-fallure configuration. 
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